This topic came up in a WCF training session (i.e. ObjectSharp's WCF course) I am currently delivering so I wanted to post a few references on the subject here:
The first is "Designing Extensible, Versionable XML Formats" by Dare Obasanjo
http://www.xml.com/pub/a/2004/07/21/design.html?page=1
The second, slightly more academic treatment is "Extensibility, XML Vocabularies, and XML Schema", by David Orchard, an extensibility guru who has much to say on the topic:
http://www.xml.com/pub/a/2004/10/27/extend.html
And then finally a series of writings by Tim Ewald which are chock full of extremely powerful Versioning advice:
First his "Services and the Versioning Problem" article in the MS Architecture Journal:
http://msdn2.microsoft.com/en-us/arcjournal/bb245674.aspx
And then a series of blog entries:
http://pluralsight.com/blogs/tewald/archive/2006/04/14/21733.aspx
http://pluralsight.com/blogs/tewald/archive/2006/04/25/22704.aspx
http://pluralsight.com/blogs/tewald/archive/2006/05/05/22961.aspx
http://pluralsight.com/blogs/tewald/archive/2006/05/05/22963.aspx
I have been doing some Windows Workflow (WF) consulting with a customer lately and I was talking to a developer who had just been thrown onto the project and was struggling with WF, not feeling that he had a solid grasp of the paradigm. No one was more surprised than me when I was able to help him so easily by simply pointing out that WF is an instance of the “Command Processor Pattern”. Because it seemed to help him so much I thought it might be worth sharing here.
Before I get to this however, I just want to state for the record how important I believe the Gang of Four (GoF) Design Patterns book is. I always assume, sometimes incorrectly, that every developer has intimate knowledge of the GoF Design Patterns book. I believe reading it is a right-of-passage for every developer, and I would always steer people towards the original book, as opposed to “knock-offs” that try to “translate it” to another programming language. If you haven’t read it, stop everything and read it!
The Command Pattern
In case you can’t recall it, the authors state the following as the intent of the “Command Pattern”:
Encapsulate a request as an object, thereby letting you parameterize clients with different requests, queue or log requests, and support undoable operations
However, I think the essence of it is well captured in the structure diagram:
The Command Processor Pattern
After the GoF authors, the POSA1 authors expanded on the Command Pattern by writing up what they called the Command Processor Pattern, a pattern that built on the Command Pattern to show how easy it is to use it to do things like undo/redo functionality etc. Again, for the sake of brevity, I will only show a single UML class diagram that I think conveys the essence of the pattern:
Compounding Command
Later, two of the GoF authors wrote a paper called “Compounding Command” in which they showed how some of the other GoF patterns can be combined with Command in interesting and powerful ways, which I’ll summarize here by once again showing a single UML diagram that I think captures it quite well:
Finally, WF as an Instance of the Command Processor Pattern
Now, mapping these ideas to WF is, I think, as easy as first of all making the following connection:
Activity à Command
WorkflowRuntime à CommandProcessor
Armed with these mappings, then showing the structural relationship of some of the key classes in WF shows very clearly how it can be conceptualized as the “Command Processor Pattern”:
The Value of Patterns
As a final comment: the GoF authors state explicitly that they weren’t doing anything new but rather placing well thought out names to reusable design nuggets that people had been using for years already. The real value in that was evident in the interaction I had this week with that developer: once he thought of WF in the context of the Command Pattern he immediately understood what I was talking about because the pattern name was part of our "shared developer lexicon".
I recently found out that the abstract I submitted for DevTeach, held in Toronto May 12-16, 2008 was accepted. I am very excited about it, both about presenting and attending because I think it is shaping up to be an excellent conference. My talk, called ``The Convergence of SOA, REST and BPM``, is in the Architecture Track (See here http://www.devteach.com/Session.aspx#91 for all the talks in this track).
I`ll try to blog more about what to expect from the talk as I figure it out better myself, but suffice it to say that these 3 pillars of contemporary software architecture are finally coming together for me, at least enough for me to want to talk about it with others :)
I am in the process of writing a new course for ObjectSharp on Windows Workflow Foundation (WF). More details about the course can be found here: http://www.objectsharp.com/training/coursedetail.aspx?id=1145. For even more information, contact our Training Manager, Julie James, at 416-216-4603 (or 1-877-So-Sharp) ext. 1 or via email at Training@objectsharp.com.
Taken along with the existing WCF course http://www.objectsharp.com/training/CourseDetail.aspx?id=1135, students will be given a hands-on guide to building a successful Service-Oriented Architecture (SOA) using the .NET 3.5 platform.
This saturday (March 1st) I will be giving a talk entitled "SOA Using WCF & WF" at the Toronto Code camp 2008. The site is here http://torontocodecamp.net/. My talk is in the .NET Framework track and the schedule for this track can be found at: http://torontocodecamp.net/Sessions/NETFramework/tabid/64/Default.aspx. The abstract for the talk is:
"In the world of SOA it is useful to classify your services into two types: Business Activity Services and Resource (or sometimes Entity) Services. In this demo-centric session we’ll look at how Windows Communication Foundation (WCF) and Windows Workflow (WF) can be used within this classification system. More specifically we’ll look at how WF is an ideal choice to use “inside the boundary” of your Activity Services to carry out the long-running work the service represents."
It looks like there are lots of good sessions so I hope to see you there.
The Visual Studio team at Microsoft is saying (see here http://blogs.msdn.com/somasegar/archive/2007/10/17/f-a-functional-programming-language.aspx for instance) they'll work closely with Microsoft Research to build support for F# into VS. Back in my University days I did a fair bit of ML so this will be like re-uniting with an old friend 
.
Last Saturday I gave a presentation at the "Okdevberfest" Code camp (http://www.okdevberfest.com/) held at the University of Waterloo. The slides as well as the sample code I covered can be found in the attachment above.
Cheers to Dave Totzke and his fellow organizers for putting together an excellent code camp.
A while back I gave a presentation at one of ObjectSharp's SMART events on "Bridging SOA & BPM" (http://www.objectsharp.com/ttdinvitation/invite070510.htm). The slides for the presentation can be found here: http://www.objectsharp.com/cs/files/folders/68276/download.aspx. One of the things I thought about saying during that presentation was that the question of whether to approach an enterprise architecture from a "services first" (i.e. SOA) perspective or from a "business processes first" (i.e. BPM) perspective is analogous to the question of whether one should approach application design from an "object-oriented perspective" or a "procedural perspective".
Now, before I elaborate on that, I'll just say that I chose to leave it out of the presentation because I view it as something that could be easily misunderstood. In saying this I am not saying that SOA is anything like OO. In fact, in my experience, the single greatest impediment to successful SOA adoption is getting people to let go of the OO mindset they have developed over the past 20 to 30 years. To be successful at SOA you have to "let go of your objects" and think more in terms of (business) documents and document exchanges. Many of the web service development tools have contibuted to this problem by making it so easy to turn your objects into services simply through the annotation of the appropriate attributes. Services should be thought of as having well-defined boundaries and, as such, stepping across those boundaries should never be taken as "lightly" as a call on an object residing in the same address space as the caller. Instead, invoking a service should be viewed more like what happens when you apply for a passport: you send off a self-contained application form (or document) and it is processed asynchronously by the receiving service. This notion of self-containment is important and is often lacking; i.e. each document in an exchange must contain the complete context for the request to be processed.
Furthermore, objects typically have an interesting life-cycle whereas services don't; think of services as simply being out there on the network receiving request messages, processing them, and sending out response messages or request messages to other services. The individual handling of a given request may have an interesting life-cycle but not the service itself.
Returning to my analogy, the key point that I am trying to make is that when looking at how things get done inside an enterprise, one can view it from a process-oriented perspective (which is analogous to looking at the software system from a procedural perspective) or they can view their assets as a set of self-contained resources and/or processes that systematically structures the various business capabilities the business can provide (which is analogous to viewing the software system from an object perspective).
Using documents as the input/output of self-contained business processes/capabilities goes hand in hand with organizing those business capabilities into discrete, felxible, self-contained (i.e. autonomous) "business services". In this view of things, the internal logic of the business process is hidden, or encapsulated if you will, and the only visible "interface" to the business service/process is the documents that can be exchanged and how those document exchanges are choreographed. That is, from a SOA perspective, internal process (or orchestration) logic should be hidden inside a service boundary and the emphasis instead must be placed on the documents being exchanged, their hopefully standardized syntax or format, their shared meaning, and the "business protocols" governing how they are exchanged.
After a two year stint at Amazon, Pat Helland is back at Microsoft: http://blogs.msdn.com/pathelland/archive/2007/05/14/i-m-glad-to-be-back.aspx.
I consider this to be great news, both for Microsoft and for those of us that appreciate whenever he shares his thoughts (via papers, presentations, blogging, metaphors, songs, etc...) with the developer community, something he is pledging to do more of now that he is back at the Big House.
I am going to be giving a talk at the next ObjectSharp SMaRT (Software Management Round-table) event called "Bridging Service-Oriented Architecture & Business Process Management". If you are interested in either of these two disciplines, or in particular in how they are related, then please join us for this upcoming breakfast event. More information about the event can be found here:
http://objectsharp.com/ttdinvitation/invite070510.htm.
Wow, all in a matter of a very short time, we see:
1.) Visual Studio "Orcas" CTP released with improved REST support for WCF (as part of .NET 3.5). Although the talk seems to be that you will have to wait for the next drop for the real meat of these improvements.
2.) Tim Ewald, Former Microsoftie who remains close with some of the Microsoft Inner circle, announces that he finally "gets REST".
http://pluralsight.com/blogs/tewald/archive/2007/04/26/46984.aspx
3.) And then, the icing on the cake, a CTP release of RESTful Data Services codenamed "Astoria" http://astoria.mslivelabs.com/Default.aspx. Here is a sample URI accessing some data from the Encarta Database:
http://astoria.sandbox.live.com/encarta/encarta.rse/Areas [ 6 ] /Articles
All in all it looks somewhat promising but what's with the non-standard URI's like "../Areas [ 6 ] /.."? I sure hope they fix that and change it to "../Areas/6/.." as it should be.
Looks like some of the people in and around the "Big House" are finally getting REST!
I am in the process of writing a new course for ObjectSharp on WCF. More details about the course can be found here: http://www.objectsharp.com/training/CourseDetail.aspx?id=1135. For even more information, contact our Training Manager, Julie James, at 416-216-4603 (or 1-877-So-Sharp) ext. 1 or via email at Training@objectsharp.com.
WCF is a very well-designed bit of technology. The separation WCF makes between the "service" itself and the plumbing required to interact with the service is very profound and allows one to have a great deal of control over a service once it is operational simply by changing configuration options. I can't say enough about how well WCF makes the separation between a service's design-time characteristics and its operational characteristics.
On the other hand, one of the other dominant design features of WCF, namely its unified programming model for all forms of "Connected Systems" development, has a way of making me nervous. Consider that there is a world of difference between:
- Middleware components that might have a fairly fine-grained interface and across which you might span the odd transaction, and
- Autonomous, enterprise-grade, course-grained services with document-oriented message schemas across which you wouldn't dream of spanning a transaction (because it wouldn't be autonomous anymore then would it?).
Yet WCF supports the development of both kinds of services, all within a single programming model! This is both very powerful and at the same time potentially dangerous. Why dangerous? Well, for many people, when given a means to define their services in their tried-and-true object-oriented paradigm the result is often not suitable for the second category of services; i.e. the course-grained, autonomous services. As such, a very important aspect of the course for me will be to highlight how to design your WCF service and data contracts to support either paradigm.
Also, for better or for worse, XML Schema is the mechanism of choice for defining the structure of the messages that flow in and out of web services. Yet WCF wants you to stay away from XML and work with its data contracts instead. However, a lot of people have spent a lot effort thinking about how to design extensible XML schemas so that your Services can evolve without impacting its current consumers (a crucial aspect of what is more generally called loose coupling). So another important theme for my course will be on how to translate some of that knowledge into WCF's data contract space and still end up with extensible, interoperable message formats.
The course is being worked on now so it is still evolving. Let me know if you think there is anything else it should include.
Today I had the great pleasure of giving a talk to the graduating class of Conestoga College's "Computer Programmer/Analyst" program, and a few students from the Health Informatics degree program as well. I must admit that at first I struggled with what to talk to them about but when I thought about when I was a student, I remembered simply being eager to know anything more about the world outside the school walls. So I decided to just give them a glimpse of the world that I know best, namely the world of Software Architecture. Since that is primarily what I do, I decided to try to give them a feel for what Software Architecture is.
I have posted the slides from the talk here: http://objectsharp.com/cs/files/folders/37913/download.aspx, but bear in mind that, without the accompanying talk, the slides themselves offer very little. However, I thought I'd make them available in case any of the students wanted to look at them to remind them of what I talked about.
I thoroughly enjoyed the experience.
I often find myself thinking that "Service Agents" play an often under-estimated role in SOA. Something came up in my current consulting contract recently and it got me thinking about the importance of Service Agents in a SOA so I thought I'd share a few thoughts I have about them here.
Agents are not just proxies
When it comes to consuming many services, at least SOAP-based web services, most toolkits provide the means to generate a proxy to communicate with the service. So the first thing to clear up is that these proxies are not Agents; at least in my view, Agents have some kind of additional smarts, typically about how to interact with the service.
There are different kinds of agents
There are the I-am-working-on-behalf-of-the-consumer kind of agents whose responsibilities are centered around making life easier and/or more effective for the consumer of the service. There can be a whole host of reasons why you need a service agent to help consumers interact with the service, but the following are some very typical reasons:
- Limited or unpredictable connectivity or simply that the consumer needs to operate in an offline mode
- Performance problems associated with service calls that necessitates things like client-side-caching, batching requests, disassembly of aggregate results, etc...
Some very good articles on this kind of agent can be found here:
There are also the I-am-working-on-behalf-of-the-service kind of agents, whose reason for being in life is to do grunt work for the service. The perfect example of this kind of agent is Google's web crawlers: they are agents out there to help Google index the web and, in so doing, they periodically upload what they have gathered to the "search service". Sometimes agents can do a bit of both.
Now let me try to share the situation I came up with recently, and how the notion of a service agent played into the solution. Consider the following diagram:
This is a generalization of the real situation but the situation is, in my experience, fairly common: you have a new service you have introduced into the enterprise, in this case a "LeadManagementService" to help marketing and sales to coordinate the capture and then subsequent follow-up of sales leads. Now it is rare indeed that you can transform the whole enterprise overnight to make use of this service but it is also less than ideal if "leads" are captured in more than one place. As in this case where leads are captured in two disconnected stores: the LeadManagementService and the CallTrackingDB.
Now consider a new version of that same infrastructure with a new service agent (in orange) in place that solves this problem by retrieving "lead info" from the CallTrackingDB and uploads leads into the LeadManagementService based on what it learns from that retrieval:
This synchronization could happen at some regular info and likely wouldn't have to be too frequent; perhaps even daily would suffice. I see this kind of agent as an "I-am-working-on-behalf-of-the-service kind of agent"; in this case I see him as being out there gathering possible leads in other systems on behalf of the service, and then ultimately informing the service of those leads.
Also, note that I use the term "agent" loosely here. I had one situation like this before and the "service" was implemented using BizTalk and the "agent" was simply a SQL Server Job Agent that was run every night, doing a file drop based on results from a particular query it made, and the batch results in the file were then picked up by the BizTalk service via a File Receive location.
Although it doesn't show it, the "Product Web Site" consuming the service might well require the services of a pretty smart agent as well. If you consider sites like Amazon, they go to great lengths to monitor every click the user is making to determine what they might be shopping after; you could easily imagine any product/service web site doing some some detailed tracking of the user's movements in order to inform the service of possible leads; and it would likely only want to submit its leads after it has analyzed the user's full session. Sounds like an agent to me.
Agents are often services themselves
In a recent post about SQL Service Broker (SSB) I mentioned that at least one of the reasons it hasn't been as widely adopted as it should be is because of its lack of interoperability, but this doesn't have to be the case. Admittedly it's non-trivial work but SSB can still be a very powerful tool when one needs reliable long-running conversations involving open web services (i.e. either SOAP-based or perhaps RESTful). Consider the following diagram that illustrates how "Service Agents" can help:
That is, the "Service Agents" in this example, which are themselves services (i.e. SSB services), are responsible for helping the "Application Processing Service" carry out reliable long-running conversations with other services.
In the end, I feel that service agents are a crucial aspect to a good SOA, and are often under-utilized.
Last week I did some training in BizTalk and of course the topic of correlation came up. I prepared a little animation to try to convey they idea behind correlation to the students taking the course and I thought it might be worth sharing here: http://www.objectsharp.com/cs/files/folders/26831/download.aspx