What *IS* software anyway?

Goofy question? Not really. Have you ever thought about what software is? Is it just 1's and 0's? Is it just the field where you are paid to toil away in? Is it a product to bring your company wealth and fame? Is it tool there to help others do their jobs, whatever they may be? Understanding that can really have an impact on what we create, how we go about creating it and how usable it is after we deliver it.

One of the valuable resources I've found out on the vastness of the web is the Software Process and Measurement Cast. Despite the unfortunate acronym chosen (SPaMCast? Really? How many people are going to click on a link that says "SPaM"? How many email filters will even let it through?), it is my favorite software related podcasts. Thomas Cagley has really solid guests from across the spectrum, asks good questions and related blog essays on each topic. My favorite episode of all time is Episode 36, which is actually the 2nd part of an interview with Phil Armour. Phil's answer to the question, "Software is not a product. It is a medium, it is a place where we store knowledge."

He explains that there are five such knowledge storing media in the world. They are:

1. DNA - where species store their knowledge
   
2. Brains - a transient knowledge read/write store
   
3. Hardware - he gives an example of a ruler storing the knowledge of length or a WW II bazooka that has the knowledge of how to aim it built into the sighting mechanism.
4. Books - especially the movable typeset books
5. Software - where the knowledge is not just stored, but it is executable. For example, software doesn't teach me to do the computations needed to compute the standard deviation of a set of numbers- it does it.
   

He has a lot more discussion about this but just the concept really made a lot of things click for me. When we think about software being a way to encode (and later execute) knowledge, it makes complete sense why long drawn out projects are doomed to fail. Even IF we could completely capture the requirements correctly at the beginning of a project, what we delivered at the end would still miss the mark. Why? In the intervening time the user would have learned some new things. New knowledge that we didn't encode.

And why do we have such a hard time getting the requirements correct? Because, most of the time, we aren't the ones with the knowledge we're trying encode. And the people with the knowledge don't know how to encode it. It is as if the only way a book could be published is if only publishers knew how to write. You know bass fishing and want to share that knowledge? Fine. Call a publisher, describe what you do and see what they come up with. Maybe, if you're lucky, the publisher will come out to actually see you in action and not just go off your description...or let you read an early draft. Oh, and when the book does come out, you now have to fish exactly the way the book describes it and only that way.

We also, all too often, misunderstand "the way we do it now" with absolute knowledge and encode it with no eye towards allowing it to change in the future once we, or the user learn something more. That doesn't mean coding so that everything is an option. But coding in a test driven style so that we have the confidence to go back and change something later and know we didn't break everything else? Try shifting from Scrum with fixed iteration length to Kanban where you have no time constraint but want buckets for the different areas the work item flows through- can your tracking software handle that?

It also tied in to some of the Real Options thinking. Why do options have value? They are giving you some freedom before you've encoded something, which is then costly to change. Time to get more of the knowledge.

Phil also has a good point about things really taking off when the medium is applied to itself. Like when steam was used to power manufacturing lines to make steam engines. How well do we encode (and execute) the knowledge of how to build good quality software into the building of software? Automated testing and continuous integration tools are a good start, but where else can we apply it?

Do you design your work?

Happily, I was finally able to attend the Lean Software Austin group's meeting this past Monday. Scott Bellware was leading the discussion on Kanban and applying it to software development. Unlike some groups where the meeting is like a lecture, this was very participatory with lots of dialog from different folks. Maybe this is because the group is still comparatively small, or since the area is still new so everyone is willing to share and try to learn from each other...no one is worried about feeling like they don't do it "right" (yet).

One of the facets of Kanban that caused interesting discussion was the idea that work units should be the same size as they pass through the work flow. This is the ideal since it promotes smoother flow and is necessary in order for the Work In Progress (WIP) limits to be effective. Can it always happen? Of course not, but could it happen more often if we tried. I believe it was Scott who said, "Design the work and design the implementation".

After the initial revulsion to Big Design Up Front (BDUF), we've come to the understanding that you don't ditch design (of the implementation) altogether but you design appropriately. That the further away something is from being implemented, the lower the fidelity of the design. In a similar way, in the Scrum world, there's also the understanding that estimates of size get more accurate for a user story as you get closer to implementation. And there is some other "design of the work", but I think a lot of it is accidental and could be improved if it was done explicitly. For example, after sitting through a few interminably long poker planning meetings, Scrum teams, or part of the team-usually the more senior members, start to sit down with the Product Owner before the planning session to "groom" or "prep" the top part of the backlog. These sessions take a look at those stories that will likely be included in the next sprint and make sure there is at least enough detail or understanding for the team to be able to have the conversation during planning. Often, the result of these sessions is the Product Owner needs to further refine his thoughts or gather more information before planning.

One benefit to being more intentional about designing our work would be gaining more learning about how we did in retrospect. Teams spend so much time in planning doing story point estimation, but rarely have I seen them spend much effort at the end of an iteration looking at how they did on those estimates so they can be more consistent and more accurate in the future. In a Lean environment, inaccuracies in sizing the work would be apparent sooner since work that is larger than the standard will slow down the flow- starving the downstream parts and backing up the upstream ones. Just like designing the implementation, we want to achieve a balance to have "good enough" design of the work. For estimates of size, that might mean t-shirt sizes (Small, Medium, Large) for work that is some ways off and more detailed estimates as the work comes closer to being taken on.

Another benefit to designing the work, beyond the area of sizing, is it gives us a chance to recognize gaps in our skill set by looking ahead, or special circumstances that might apply. The larger the gap, the earlier we should look out how we're going to fill it, maybe by bringing in a specialist on a contract basis so we can learn from her, getting some training, doing early experiments or some other option. Special circumstances might be recognizing that a piece of work might be better done before someone on the team leaves for vacation. So, the work could be moved up in the order to take advantage of his special knowledge or skills.

How much time does your team spend designing your work? How does this compare with the effort in designing the implementation?

For those of you in Austin- hope you make it to the September Lean Software Austin meeting!