Semantic Werks

I just watched an excellent presentation by David Anderson at the Agile 08 conference. He talked about moving agile to more complex, enterprise-scale projects, and how the current agile practices work.

My big take-away was his characterization of requirements as ‘perishable’. In his view, unmet requirements are like unsold inventory, essentially liabilities until they can be turned into working systems. He borrows from the just-in-time philosophy of Lean manufacturing to emphasize how unsatisfied requirements ought to be taken off the shelf and implemented as soon as possible.

In his view, future software/system architecture decisions will be much higher-level, business-value decisions. He mentioned software product lines as an example: the architect is someone who can separate product variability and encapsulate common behaviour, rather than over-specifying lower-level designs. There will be an increased need for ‘customer intimacy’ to generate unique systems that capture a core competency or competitive advantage.

There’s a lot more in this talk, including some interesting ideas on community evolution, so it’s well worth the time to watch.

At RE08 in Barcelona, Michael Jackson mentioned this book as a must-read for people in requirements engineering, and respecting my elders, I dutifully picked it up.

The first contribution of this book (most important?) is to highlight that engineering (and I sort of unthinkingly lump software engineering in there, for the moment) is NOT just the application of scientific principles to real-world problems. They are ’separate spheres of knowledge’.  Anyone who has tried to construct the most rudimentary technology — e.g., a tree-fort, change a tire, etc. can verify this. While on paper the solution is obvious, in harsh light of day, the nail doesn’t go in deep enough, the lugnuts are seized, and so on.

I found the book often went into excessive detail, of interest only to aeronautic buffs, but there were sections at the beginning and conclusion of each chapter that were highly interesting. Perhaps most relevant to my research, and to software engineering, was the chapter on aircraft qualities. Qualities include aspects like how ‘heavy’ the control stick is, how stable the plane flies, etc. Anyone who has driven several different types of car can relate to these properties, I would think: does the car shimmy at high speeds (like my first car, an 89 Toyota Tercel)? Do the brakes feel squishy? Does the door have a convincing thunk when closing, or a tinny clang?

For early aircraft design, these qualities were secondary to functional aspects: ‘With the essential proviso, of course, that an airplane can be reasonably and safely flown by the pilot, its utility depends crucially on these other characteristics [speed, range, ceiling, carrying capacity ...]‘. He goes on to add that only when these functional properties were largely satisfied (and here I assume he means that there were diminishing returns on further research into these functional characteristics) was flying quality considered. Eventually, specs for new aircraft came to include sections on flying qualities, encapsulating ‘the notion that desired subjective perceptions of pilots could be attained though objective specifications for designers’. Because these qualities were subjective, the specs tended to be design guides to assess tradeoffs rather than objective, measurable requirements (much like in NFRs and software design).

Vincenti argues that engineering knowledge grows via an evolutionary variation-selection process; as a result, ‘technology’s failures require examination as well as its successes’. The important lesson is that approaches that attempt to design something perfectly from the beginning seem doomed to failure. The best we can do is prepare to throw one version away, until we have a good body of evidence to support one particular solution. For example, NASA tries to re-use Mars expedition software and hardware as much as possible; e.g., the particular circuit board, the software language, and so on. The reason for this, as Jackson explained in his keynote, is to leverage the benefits from “Normal design” and specialization.The characteristics are known and well-tested; keeping these things constant allows for incremental specialization at the margins (e.g., improved science packages, new rocket engine, etc.)

My take-away is that the most important thing to understand in approaching a new problem is what pieces of that problem are normal and what pieces radical. Focusing on the radical is key, because we can assume this is where the failures and iteration will need to happen.

The study of the history of technology is one of great interest to me. I’m particularly interested in software engineering history, namely, how software-mediated technology has arisen, the forces that have shaped the solutions, and so on. I’ve written one paper on this, its subject the history of distributed computing (REST, RPC, CORBA, and others).

There’s a good general-purpose journal: Technology and Culture, which addresses a wide array of topics. There is also the Computer History Museum’s Software Industry SIG, and the IEEE Annals of History of Computing, but that’s about it. The focus also often veers into straight-up narrative, rather than a critical look at how/why certain things evolved.

I think it’s pretty clear why history matters, “those who forget ..” etcetera etcetera. An interview I watched the other day made the case that the inventor of LINQ, Anders Hejlsberg, isn’t ‘inventing’ so much as engineering: applying to Visual Studio and .NET the ideas behind LISP, Python list comprehensions, and so on. Of course this is still creativity in action, as Steve Martin would say. Because Hejlsberg understands his history he is able to leverage it (whether because he is well-read, was there for it, brilliant, or all of the above, I don’t know).

Here is the point, however: good history is not about dates and people, the ‘what happened’ narrative aproach, but ‘why‘: why REST, why Apache took off, why Perl has faded, why WS-* is despised … As you can see, there’s a lot to look at.