The SOA and Agile Culture Clash

In SOA and Agile: Friends or Foes?,  Amr Elssamadisy opens a discussion on the perceived disconnect between what should be the mutual admiration societies of SOA and Agile developers.  Most of the comments to Amr's article are also informative and and well-written.  However, in the discussion, not mentioned are the non-technical forces in play which, hopefully, this response can illuminate.

The discussion about SOA-versus-, or SOA-plus-Agile is about origins and culture as much as it is about the what (SOA) and the how (Agile practice).  Lines have formed between the teams of developers building incremental improvements to corporate IT assets and the smaller, more independent teams behind the lightweight, Web 2.0 style of applications like BaseCamp and Ta-daList.

Though the two don't require coexistence, SOA is the glorious 'what' for Rational Unified Process (RUP) 'how' folks.  The authors and talking heads pushing SOA seem like the same academic, RUP-tower individuals that heavily influenced how software, supposedly, should be written, at least, up until the late 90s.  Lots of grandiose prescriptions and no code demos showing how real problems get solved.

Pragmatists like Kent Beck, DHH (Rails creator), and others sense the happy RUP+SOA marriage and run as fast as they can the other way in order to build what people actually want, not what the process says they do.  But, unfortunately, this also happens to discard whatever goodness there is in using modern SOA as a useful template for defining 'just enough' design. 

And, to be sure, there is plenty of goodness in it.

But agile zealots running scared from SOA-speak, though somewhat accurate, is, admittedly, a bit of a caricature.  While it does shed another ray of light on why SOA and agile communities don't seem to align, the truth is that hard working IT architects and programmers regularly combine architectural decomposition and iterative development (with lots of feedback) effectively.  The SOA and Agile camps just don't always call these things by the same name.  And, typically, they're solving entirely different classes of problems.

While at webMethods, I heard quite a few first-hand customer stories of massive SOA rollouts that did so two weeks at a time.  Tons of small focused meetings ensured that things were progressing effectively.  Each two week iteration focused on a well scoped business problem.  Bit by bit, their tactical, agile approach (though maybe not acknowledged as such) yielded increasing strategic benefits when, with increasing frequency, newly created services would call those that had been created in iterations past.  webMethods frequently measured the ROI on efforts like this and wielded this data in sales presentations to future customers.  From what I saw, Agile SOA is alive and doing quite well.

On the other hand, we have the Web 2.0 folks.  These are the agile purists who build focused, user-friendly applications, often written with PHP, Ruby on Rails, or something like it.  For small applications, the creator (or two) envision a solution and, using community-driven conventions, quickly layer their domain logic on top of a crisp MVC template.  Deployment involves scaling web servers in some standard way that usually includes Apache.  Voila, another node in the exploding network of software services is born.

What's interesting about the 'agile' folks and their unofficial enemy, SOA, is when mash-ups come into play.  These are those web applications (2.0 or otherwise) that include data and behavior, directly provided by yet another web application that had rendered some of its services. 

As of mid 2007, the clear trend is to provide these types of mash-up services to clients RESTfully.  WS-*, SOAP, and sometimes even XML are deemed way to heavyweight for these kinds of integration needs.  Some SAAS apps like Amazon Web Services, at one time, provided both SOAP and REST interfaces, only to drop the SOAP version. 

Interestingly and not without a mild level of controversy, even DHH now seems to be pushing the development of all aspects of Rails applications with RESTful conventions.  The result is is an application that can (optionally) serve up any of its functionality as part of broader inter/intra-net... SOA (ok, call it a mash-up if you like :^)

The SOAists and the Agilists aren't so far apart after all.

So, what does all this mean to someone who, like myself, must straddle both worlds?  Who knows.  What I do know is that it's a good start just knowing what the spectrum is, with Agile+REST on one side and SOA+tight RUP iterations on the other.

It's also useful to know that, depending on the type of problem you're solving and the community you'll be working with to solve that problem, you'll find legitimate reasons to think SOA or, perhaps, Agile first.  Either way, you'll want to look past the culture clash and effectively leverage both.