Here at Inflecto Systems we are currently bidding for a project which will be a replacement for one of the first major software systems I developed after leaving University. At the time I was working as senior developer for a company that had previously specialised mainly in web design.
Why the change?
The system in question has served the business reasonably well through a period where the company has grown at a very fast pace. The nature of the companies work means that the business processes that the original system captured also changed regularly and the system was too inflexible to easily cope with these changes. Some of the changes had been made to extremely tight unmoveable deadlines due to external pressures placed upon the company which has sometimes meant that we have cut corners to ensure our client can meet their commitments. These facts mean that over time the system has become extremely fragile and hard to maintain hence the consideration to start from scratch.
Capturing business rules and processes in a system when the system is new is easy when compared to maintaining and evolving these rules as the system changes. This would not have been a surprise to me even when fresh from university but the reality of managing some large software projects over a period of time and possibly being involved with putting my first software project out to pasture allows me to consider this from a different perspective.
The Original Software System & My Story
The original system in question did involve a very simple workflow engine which was built to allow the possibility of extension of the system. This was a simple SQL based system that allowed processes to progress through a number of stages including branching with operations working in parallel. Looking back our design was a little contrived. Our original design for moving work through stages to model the business processes although simple would have worked just fine. The missing part of the picture is that the processes that existed in the workflow were not isolated. In fact the way we went about developing the system meant that the activities were so tightly coupled that it was almost impossible to change the sequence of events or easily remove business processes. There was also no way to version workflows, have different portions of work follow a different workflow or to manage what happened to workflows already in progress when you wanted to change the activities within the workflow. All this meant that our design never satisfied the goal of allowing changing business processes to be captured by the system.
A few years after the completion of this project I left to work for a large software house and got experience of working with their workflow system. This system had a bit in common with my first system in that it was mainly SQL based but was much more complex. Essentially this system worked quite well for what it was used for. In the system various actions performed by the user called in to the workflow system and then various configurable consequences could be scripted using SQL. If I remember correctly it also tied in to a document creation engine and this seemed to be its main purpose. Overall this system was functional but it still seemed very clunky and although the company stored vast amounts of documentation on the systems I am a big fan of systems being self documenting through the use of good comments and alike.
After leaving these two companies to start up Inflecto we have never tried to integrate any kind way of easily facilitating evolving business processes within our systems. It has been something I have given a great deal of thought, however, we are mostly dealing with smaller scale systems with tight budgets and I have never felt that there were any technologies that gave us an easy route to embedding this functionality without having to crack what is quite a difficult problem from scratch. This has meant we normally end up hard coding business rules within our systems.
Moving forward with a Workflow Foundation (WF4)
I have taken a great deal of interest in Workflow Foundation (WF) from Microsoft since its release as a possible route to building an architecture to solve problems where you need to model complex or rapidly changing business rules. The first version got me quite excited but when I learnt that this was going to be completely rewritten from scratch I held off using it.
WF4 the latest incarnation released with version 4 of the .NET framework has been out for some time now and I am ready to finally jump on board and see what we can achieve with this technology. We have been using WCF which plays very nicely with WF4 for some time to write simple Web Services. I have been hugely impressed with WCF and would suggest that Microsoft is head and shoulders above everyone in terms of this technology I am hoping that I will be just as impressed with WF4.
I have been spending a lot of time looking at how I can possibly develop a solution that can satisfy my client’s needs using these technologies and over the coming months would like to share some of my ideas using our blog.
WF4 looks like a great platform and there are other tools such as the WCF Routing Service and AppFabric which offer a great deal of functionality such as:
- A graphical tool for creating and editing workflows to model business processes. These workflows can produce XAML which can be loaded at runtime allowing workflows to be easily changed. The graphical designer can also be re-hosted allowing your end users to change workflows for themselves.
- AppFabric brings decent monitoring systems for monitoring workflows and WCF services that are deployed in IIS.
- You can think of the WCF routing service as virtualisation for your WCF endpoints. The routing service can be deployed in front of your services and can then route the messages to different endpoints by investigating the contents of the message.
Having said that there are still significant problems which we need to overcome:
- Automatic location and deployment of services – We want our customers to be able to create workflows and assign various work streams to them. We need to offer them the designer and a simple way to deploy there workflows to IIS and get them hooked up to AppFabric. We then need these workflows to be discoverable from the other components that need to call in to them.
- Service versioning – We need a way to manage different versions of a workflow. We also need to be able to upgrade active instances of a workflow which have state to a new version. This is quite a tricky problem that has been discussed a bit by Ron Jacobs over at Endpoint.tv.
Leveraging A Service Orientated Architecture (SOA)
I am also very interested in looking at how we move our collective mindset towards developing Service Orientated Architecture (SOA) applications.
I will be beginning to investigate some of these scenarios in the beginning of August and hope to write a whole series of posts to share my findings so please pop back to read how we get on and more importantly what we learned. In the meantime if anybody feels like sharing useful resources or books you feel we should be reading those suggestions would be gratefully received.