Putting the 'ST' in REST

Starting to implement REST has led to some interesting discussions and research topics with regards to what it means to be "REST". After reading Roy Fielding's post "REST APIs must be hypertext driven" - I think a walk through of this point might help things (both for my own understanding and for anyone who might be reading).

Unfortunately, Roy's "rant" comes off a little theoretical - and I think that might be one of the reasons why so many people are missing the point he is trying to make. The basic idea is that the API itself gives you all the information for the route your application can go. All the "application" knows is a root address - everything else flows from the choices the user is given and what they choose. In API land, this is a very foreign concept. Normally, when interfacing to an API, the application consuming the API has a structure and pages hardcoded, and those pages have buttons which restrict the flow for the user. So the API is just a wrapper around the data that the application will be using.

Web vs API

The real difference here is how web pages work versus how the typical API consumption works. For example, if you want to place an order on a website, the flow is something like this:

Webpage Application Example:

User goes to the root URL (http://www.awesomeproduct.com), and is presented with links to product pages of stuff on sale and a search box of some sort.

The user types in "green widget" to the search box and hits the "search" button.

The user is redirected to a product search result page with all the green widgets listed.

User clicks a product result link and is taken to a product information page with a "add to cart" button.

User likes the product, clicks "add to cart" and is taken either to the cart or the general search page

For web pages, this all works because there is an understanding that you are using a web browser and are expecting HTML to come back from nearly every resource. Your browser negotiates with the web server and gets an HTML representation of the information to display and displays it as text/images/buttons/forms. The returned page may contain links to other pages or related information and you can choose to browse to that information. The page could conceivably have a form that you fill out to send a message or something.

Now compare this to an example ordering application consuming a typical "secure" non-REST API:

Proprietary API Example:

User opens a pre-built login form in the application. This page has a username/password pair that get encrypted and sent to a Login method in the API via a socket connection, which returns a session token for subsequent requests.

The application then displays a predefined search page to locate products to order. The user enters "green widget" and hits search (passing the token).

"green widget" is sent to the SearchProducts API method and it returns a list of products matching "green widget".

The application displays the list in a table and the user hits "Add to order" button on the app. This triggers the application to package up a new order object and send it to the API's OrderCreate method.

An order is created on the backend and a success message is returned

As you can see the flows are very similar - but there are some important differences which will be discussed in a minute.

Lets go "REST" !!

This is pretty much how things are done for most applications, but then some ambitious programmer comes along and says "Lets make our API REST so we can use the same API for multiple apps" (or some such reasoning). The sales people chime in and say "Yeah, lets go REST because everyone is clamoring for that buzzword" and so the decision is made to do REST. After the 10 minute "Google Tutorial" on REST, they implement the following BAD implementation:

Login method:
GET to www.awesomeproducts.com/api/v1/login?user=x;encryptedpassword=y
Product search:
GET to www.awesomeproducts.com/api/v1/products?keyword="green widget"
Order create:
POST to www.awesomeproducts.com/api/v1/orders

The developers of the app then change over their calls from their proprietary calls and responses to the new "REST" versions using XML defined in an XSD and the app is declared REST compliant! The problem here is that it is NOT REST and it is really no better than the previous proprietary non-REST version before. All that was really changed was the formatting of the data going to and from the backend, and the standardization on a communication protocol (HTTP).  Whats worse, is that it gives the appearance of REST, but it is not really REST at all.

Problems with RE"ST"

So why is this bad and how is doing it "properly" going to help anyway? Let now add a simple change to the situation to make the problem more clear - lets say management says that they want to display an advertisement for the new and amazing purple widgets after a customer logs in, but before the search. That will surely drive sales and we have to have this new feature yesterday.  So the app designers go to work and change the application to stick in this new advertisement (because changes to the API can't really make an ad show up in the hard-coded flow).

BUT WAIT - why does the application have to change?!   That is the problem with this implementation of "REST"!  The issue here, is that if it was REST, the application would not have changed to accommodate this new 'page' in the flow.  To put it in perspective, imagine that if Amazon or Google wanted to change their pages, you had to upgrade your browser to see the new pages!!  That would make the web unworkable.  Yet this is what purported 'REST' designers have been doing and the type of thing that prompted Roy Fielding to go off on his well-deserved "rant" post.  With 'proper' REST, you should have been able to insert or change the flow by simply changing the transitions returned from the API server - not changes in the client.  What you have here is what my coworker called "REST without the 'ST'", there are no state transitions being provided by the "REST" server.  Its akin to web pages with hard-coded pages and no way to link between them - you would just have to know all the pages to go to and manually type them in (or have them hard-coded in your browser) - and any changes to the web page would require you to alter the browser.

The brilliance of REST is not with the format of the data, or the HTTP negotiation - its with the transfer of state control from the client to the server.   You are allowing a third party to not only provide you with information in a format you can understand, but to control the valid paths through the "application" by letting it tell you were you can go - and then letting you choose.  Prior to REST, it was just a program that was hard-coded to known resources on a server and to find new information, you had to re-code the program to access those resources.  After REST, programs let you connect to servers which provide you a list of information(resources) you could go to.  You could then "choose-your-own-adventure" through the resources and, more importantly, the information providers could control your experiences as they saw fit - directing you down whichever back alley your choices lead.  This paradigm shift cannot be understated and is what gives REST its power.

Aside: If you want to read more about this shift, it goes under the more technical name HATEOAS (Hypermedia As The Engine Of Application State)

Roy was justifiably upset to see that great idea applied to situations where all that was "REST-a-fied" was the format of the data and some details around the protocol.  Which is really the same as the Proprietary API Example I provided above.  You basically hard code a program to pull resources in a predefined format from fixed resources.  The only thing that was really added was the standardization of format - which does allow others to more easily pick up the API, but its not REST.

So How do We Do It More RESTful?

So this is the point where most REST folks (myself included) start getting flak about being "REST purist" and that providing the data in XML via HTTP using a "REST" URL is good enough.  But you miss the entire crux of what makes REST so great - transfer of state control from the client to the server.  So what changes in the above example to make the Proprietary API Example a REST API example?

Because this post is running long, I will make a new post explaining an example of how to use REST in an API context.