Shared Understanding and/or Evolvability

(This is belated and too short. All you fast-typing bloggers out there can just be patient :) Sorry for the delay!)

How do distributed systems both cooperate and evolve?

That is a subtly different question than JJ asks:

when no human is in the loop, you need a shared understanding between the resource consumer and the resource (provider)

The only viable choices JJ then describes (WSDL or WADL) do indeed provide a partial solution to coding shared understanding, but not evolution.

The short (flip?) answer to my leading question is to create a new media type that describes the semantics necessary for both consumers and providers to communicate.

In the case of exposing a Job Application Service as a RESTful provider that could mean the following:

• for a human provide an HTML response representation with forms for review, cancel, submit interview.
• for a machine provide an XML or RDF (or something) response representation.

That machine representation must have a lot of things going for it. It must provide just enough shared information to do useful things, but not prevent service evolution. In particular, the response must:

• provide links/conformance to some shared schema type(s) (to share semantics)
• be extensible (in the "does not understand" sense)
• the shared schema can't pre-define URIs
• the shared schema can't pre-constrain all of the transition paths
  

That last one is really fuzzy, but I'm trying to express the idea with my "signposts" metaphor. Getting something done (i.e. changing the state of something) isn't alway a single step, and how many steps have to be followed isn't a very stable property. In the case of job applications, getting from "offered" to "accepted" might take a few loops in the 1.1 release of the service, or maybe the "review" state gets split into 2 steps, what happens then?

The current conception of "shared understanding" is "shared interface". In the Job Application example that means the client service is encoded to expect after "Submit Review" that the service moved the job app to "reviewed". If the client service instead was coded to have a pair of (current job app data, desired state) when the service changed to a 2-stage review process the clients would likely continue to work:

• Client: Get job app 123
• Service: job app 123, submitted, "Submit Review", "Cancel"
  
• Client: "Submit Review" for job app 123
• Service: job app 123, review1, "Submit Review", "Cancel", "Reject"
• Client: "Submit Review" for job app 123
• Service: job app 123, reviewed, "Submit Interview", "Cancel", "Reject"
  

Understanding how to encode an imperative SOA static model to a conversational document interchange is not really that well understood yet (certainly to me). I think someday it will be, but right now there is just the quotes from the REST thesis I commented on and a reference to "data reactive programming" from Roy Fielding, and I just found Mimosa.

Just need to make sure of something about the Job Application Service example: is this service the provider or consumer of the lifecycle of a job app? It clearly has an internal state machine for job apps, but what other machine process does that get shared with?

[1] Some discussion (still searching for link...) hinted that WADL could be simply returned like an HTML Form, and not used to statically generate code. I think avoids the coupling of code-gen, but I'm not sure how it solves the problem of shared understanding either.