Rock Soup, It Works!

At my current place of employment, we fail the Joel Test. We used to score exactly 0 (ZERO) on this test. At first, I thought this merely annoying, but as time passed I felt I had to do something about it. I was often distracted from what I should have been doing to deal with something that resulted from not having at least some of these processes in place. The tipping point was when I was trying to work too often from home, and I was having to copy the code and the database to my USB drive, and then install both on my home system before I could get any work done. Having freshly read The Pragmatic Programmer I thought I would give some of the techniques in that text a whirl.

My primary tactic was what they refer to as "Rock Soup." This is the practice of planting the seed of something better. You galvanize the actions of the group once they have a focus and can see the light at the end of the tunnel.

The first step was task and defect tracking. I, again, took Joel's advice and just starting using a spreadsheet. At this point, it was just me and another programmer working in a single office, so he saw me using it to record the things that we had to do, when we needed them done by, and just little tidbits of information. He started to have me label things in there by who was going to do them. Then we needed more columns to describe the nature of our tasks. Then we needed a more powerful system. He had already started a system that would allow us to make notes in our development product - a precursor to letting our users make notations in that same system.

Asking for forgiveness is easier than asking for permission. If you know something is right, just do it that way. Nevermind the boss who is breathing down your neck to get out a product six months before it could realistically be functional. Nevermind that you have several days worth of programming to finish this afternoon. Do it right. Later, you will be happy you did, when you spend less time testing, debugging and maintaining that very same code. Your boss will be happy too, because there will be less bugs reported against that code, and more customer satisfaction before they have to report a bug.

In this spirit, I made alterations to the task list system. I added various fields, split the tasks into sub lists, made the interface easier to use. This inspired my coworker to do the same. In the end, we had a fairly functional defect tracking tool that would allow us to print out reports for management to review. Could they understand what the bugs each meant? Doubtful. Did they love getting the report and watching the count move up and down? Absolutely.

Things started to get harry with the source code. In the past we had tried to use Visual Source Safe. It wouldn't allow us to work from home, and as the project dragged on, it appeared we would be working more and more from home (weekends, evenings, mornings, etc.). We needed a better solution. We asked the people in charge of the project (5 of them, 2 programmers - see a problem?) if we could get a better version control system. They agreed once we made it clear that their goals were aligned with ours, namely us working on the code easier. So, I research the best alternative for us (in our case, it was SourceGear's Vault) and we implemented it. A couple of weeks go by, and I start to get worried because our temporary license is about to expire. I talk to the heads of state, they are interested by I get no action. I buy the software. I install the license. I send them an expense report. This is normal operating procedure at most companies, but you would think that the company would be a bit more interested in protecting their quarter million dollar investment (in man hours alone) with a ~$600 version control system.

I'll stop here, because I am sure you all get the point. Just do something, especially if you know it is the right thing to do (like having version control software, or testing!). Get permission after it is done. Get your coworkers to buy in by doing it yourself and showing them how much better it is. Rock Soup really works.

Thoughts on programming languages

This article is reconstructed from some notes I made to myself on GMail (yeah, sadly, I write myself emails because I forget things). This is the sort of things that starts happening to you when you spend six weeks not sleeping. You start to wonder if you will ever get to sleep again. Then, it dawns on you, if you could only get more done, then you would get to sleep more than 45 minutes every 48 hours. What happens next looks like the rants of a madman.

madman rant starts here:

Languages manipulate meaning.

Languages are a leaky abstraction for meaning. [note: see Joel Spolsky's article on leaky abstractions.]

Therefore, programming languages themselves for[ce] the user to think in terms of the language [note: Is this a form of Sapir-Whorf affect?] because they are by definition a leaky abstraction that has to be worked around.

What is the easiest way to manipulate units of meaning?

How can we increase the density with which we maniplate meanings?

Mostly, we assign a meaning to a symbol, and then use that symbol to stand for that meaning. [circular login when you are tired, anyone?] Examples:

14 + 35 = 49

the "+" means "add them together", and the "=" means "the result is". Most of the human race has agreed on this symbology [okay, "symbolism" but you should all see Boondock Saints].

this could be rewritten a million different ways

+ 14 35

or

(+ 14 35)

or

(add 14 35)

or

(apply + '(14 35))

Because of the engrained nature of the add operation in our minds we have no trouble coupling that symbol to that specific meaning. It is when we get into something like english that we have trouble:

Like: "Jane is blue."

Does that mean Jane is an alien, or does it mean she is depressed? This ambiguity doesn't work well for computers. They want specific instructions. Therefore we need a language that minimizes the ambiguity (eliminating it would make it math, not language)

How can we minimize ambiguity? What tools can we give the developer to minimize ambiguity?

we need a structured way to write down algorithms and compositions of algorithms. In English and programming languages we use context to indicate some of the information and to reduce the possibility of ambiguity [You like how punctuation and capitalization go out the window when you are tired enough?]

Normally, we could build a hierarchy of concepts which gets us from very simple starting concepts to very complex systems like TCP/IP.

These layers of abstraction further obscure the actual manipulation of atomic units of meaning, creating further leaky abstractions.

All of programming is based on this concept. Create an operator out of the operators that you have already been given and work from there, all the way up the chain to whatever abstraction you need to stop at in order to get your job done.

If we cannot get rid of ambiguity, and we realize that all programming languages are based on composition of basic operators, then it seems that a language would be more and more useful as it became easier and easier to compose the operators together, along with their data. Also, ease of manipulation of these compositions is a factor (think macros).

the more meta information we have about an operator, the less the operator has to actually show in code. Back to the "+", everybody knows what this means, so general meta knowledge about "+" is very, very rich in individuals. We either have to educate individuals to the level that they understand "+" at, or we need a formal language for describing the meta data of the operator.

How can we minimize code? By identifying patterns in meaning, and using a single operator to express that pattern.

End of madman rant


Alright, I have slept some since I wrote this. I have let the ideas roll around in my head. I am rereading it as I write this article, over and over. I admit these assertions are based on accepting a string of postulate based (as in, I can't prove any of the above statements are undeniably true) arguments. However, I do think it brings us to an important point. The point is hidden in "identifying patterns in meaning, and using a single operator to express that pattern."

Most modern programming languages (I cannot speak for all, but the ones I am familiar with) concentrate on the expression of algorithms. There is even an algorithms class in most undergraduate computer science programs. It seems algorithms are central to the creation and discussion of programming. Algorithms are made up of units of meaning. We just assume units of meaning because our human brains are used to assigning meaning to a symbol (written) or a sound (verbal).

To my knowledge nobody has given any thought to finding patterns in units of meaning. If, as my sleepless mind postulated, we can find regular patterns in the use of units of meaning, then we could assign a symbol to those more generalized patterns, allowing us to write more succinct programs using the new symbolic language. Think of it as writing a function that takes another function as an argument along with the arguments to be operated on. We find the general patterns of execution for types of meanings in human language, assign them a symbol, and then use those patterns with specific meanings to generate actual code. This would significantly increase the density of the units of meaning, because you are implicitly placing a single meaning into the context of a more generalized pattern. It requires less symbols to express more meaning. More succinct expression is more expressive power. Paul Graham agrees.

Is this possible? I have no idea. Am I going to try? Absolutely.

[Note: If I am completely daft and somebody has already gone down this road and come back again, please email me with a reference to their work. I doubt that I am the first person to consider programming languages from this angle, but I am ignorant of other's attempts.]

I was wrong, wrong, wrong...

On my website I posted an article discussing the relative power of programming languages. I stated that it was an unfair test by Paul Graham to determine the power of a language by how it handles closures. I was wrong. I admit it, I was wrong. I also talked about how I was unsure if other languages offered the same functionality as reflection in C#. I was wrong about that too. I am o for 2. Let's see if the third time is the charm.

You're probably thinking (I am) "why am I reading this guy's blog if he is wrong all the time?" The answer is straightforward; I learn from my mistakes. Paul Graham was right, a closure is a good test of the power and complexity of a language. How did I learn my lesson? Ruby.

A Ruby closure:

def closure(num)
Proc.new {|x| num += x }
end

This code carries the value it is initialized with as part of its local scope, which means that you can accumulate values by adding successive numbers. [Note: I should also point out that this code works as an accumulator for strings as well, due to Ruby's duck typing. In fact, it would work as an accumulator for any class that defined the "+=" method.]

Paul's solutions page supplies an additional implementation of the same functionality for Ruby. The brevity of the code is what is paramount in this example. It is so easy and straight forward to write an accumulator in Ruby, particularly when you compare it to C# (convoluted C# example available in my original article, but trust me it is much more complex and much more wordy). With .NET v2.0 in full swing, I am pretty sure that part of this problem is mitigated by anonymous delegates. While I haven't written and tested any code to support this assertion, from what I have briefly read it appears as though it is much simpler to solve this particular riddle.

Where does this leave us? In my case, the issue is not straightforward. At home, I can work with Ruby. At work I am still using C#. I am the only programmer at my job that has time to learn new languages and compare them in a theoretical sense. If I start writing support programs at work in Ruby, I am certain that there will be complaints because nobody else will be able to edit them without learning Ruby (although, seriously 20 mins?!). So, I am stuck using C# for most things. Not to say that C# is a horrible language, but it is not nearly as powerful as Ruby, or Lisp, or Python, et cetera.

How do I get around this? That question will for the basis from which the next few blog entries are to be written. I have every intention of exploring my varying options. How can we write more succinct code in C#? What techniques can make things easier and faster? I will be exploring various forms of code generation, code reuse, object patterns and C style preprocessors.

Most importantly, I know I am not alone, so the solutions I come up with will be posted on this blog, with full code to back them up. I am not above taking ideas from others and making them my own, as long as it gets the code I need written, written faster (proper credit will be given to any individuals who knowingly or unknowingly contribute to my obsession...I mean goal.)

[Side note: seriously, if you haven't learned Ruby or given it a look, please do. Once the business guys hear about it enough (GO RAILS!) we will start to see companies using it to actually do business. Once we see that, then we can have the chance of working in a language that we would use at home, at work. I am not biased either, I think it would be much more efficient to write code in Python than in C#. I think I am just tired of declaring my variables.]