Discussion: API versioning

[wgorder]

This popped in my head, and I am not sure if there is anything here or not, or even if this is the right place for it.

With cloud native applications being the way things are moving, I find myself needing to think through better approaches for versioning our API's. Found some good thoughts here:

https://github.com/restfulapi/api-strategy http://www.baeldung.com/rest-versioning http://www.lexicalscope.com/blog/2012/03/12/how-are-rest-apis-versioned/

I can see there are a few common strategies out there with a few perhaps being more "proper"

I am wondering if we cant do something to make this easier using Spring? Factors to consider might include:

• Support multiple versioning strategies
• Caching
• Swagger support
• Maybe a way to show the usage of one version over another
• Dependency graphs (not sure if this fits here but maybe a different feature)

While perhaps these are not all things that are terribly difficult to implement without a framework, can we make it easier? It seems there are some opportunities to abstract out some common things. It may also help move people to think about this more as they develop for the cloud.

Thoughts?

[spencergibb]

We have the start of a dependency graph contributed by 4finance. Currently for zookeeper, but we'd like to generalize it. It seems like there is a general dislike of swagger, something like spring-restdocs seems to be preferred.

[wgorder]

@spencergibb glad to hear there is a start to the dependency graph.

As to swagger, I am by no means married to one implementation over the other. I am more just looking for some sort of API doc functionality. If we did implement some sort of API versioning feature it would need to work with one or more of those libraries.

@mstine was on site where I work a little while back and gave a presentation. There was a slide with this death star of microservices with lines going everywhere between them. Of course the question is always asked how can you deploy one of those without breaking the others? Of course strive to not introduce backwards breaking changes in your API's is always the goal but its not always possible. That leaves API versioning.

Spring cloud/boot, Lattice, CF all work nicely together to address a lot of the common needs. However API versioning while seemingly necessary is not really called out in any of these projects, as of right now. The purpose of this issue is to investigate as to whether there is a story/use case for that or not. Can it be made simpler than it is today? I put in in spring-boot rather than spring-cloud as I can see API versioning something that may be desired outside of the cloud as well.

Some people have already tried to hack together various implementations like this one:

http://stackoverflow.com/questions/20198275/how-to-manage-rest-api-versioning-with-spring

The 3rd link in my first post shows some SO posts that show there is quite a lot of interest and opinions around the subject. I think this is something that is going to come up more as we move to cloud native applications.

[wgorder]

Still exploring this a bit. Here is a pretty good gist of some of the ways you might do it now if you were versioning your resources:

https://github.com/mindhaq/restapi-versioning-spring

Its kind of ugly, and I can see it growing to be rather unmaintainable if you were unable to remove old versions.

Here is someone else's stab at improving it: http://java.dzone.com/articles/rest-json-service-versioning-%E2%80%93

I am trying to figure out if there is an elegant way to do this or if its just necessary ugliness.

And of course a long post explaining how there is no right way (with lots of additional wrong ways in the comments) http://www.troyhunt.com/2014/02/your-api-versioning-is-wrong-which-is.html

The same types of questions apply to HATEOAS, whats the right pattern there? Seems the question was asked with no response. https://github.com/spring-projects/spring-hateoas/issues/285

Then of course there is Roy Fielding saying don't version your API at all http://www.infoq.com/articles/roy-fielding-on-versioning

The more I look, I am starting to think this feature idea may be a lost cause. There seems to be no agreement on this topic to be found anywhere.

[dsyer]

I'm not sure what the new "feature" would be really. All the tools exist to publish HTTP "APIs" and to control the content with various different strategies. Content negotiation and HATEOAS are really the only sane ways to handle changes in HTTP resource contracts IMO (and in the opinion of most of the Spring Engineering team if they have one), but there are plenty of people who obviously disagree, and for expediency's sake I can't say I blame them in a lot of cases. Anyway, I digress. Since there are tools available for content negotiation and link building and documentation, they might not be perfect, but if not the place to fix them is in their own code, not in Spring Boot or Spring Cloud. Nevertheless, if anyone has a concrete proposal for a feature we can consider it here or in one of those other places. Or is it really just a documentation issue (a blog series or something)?

[benneq]

I would be nice, if Spring (in general or Boot) would provide some convenient API versioning feature.

There are basically only 3 ways to do API versioning: 1. PathVariable, 2. RequestHeader, 3. RequestParam.

1. PathVariable: Difficult to guess where the developer wants to put it (e.g. /api/v1.2/foo or /v1.2/foo). Also this is not HATEOAS friendly, because URLs represent resources and /v1/foo and /v2/foo look like different resources, but they aren't. They are just different representations (more or less like a projection) 2. RequestHeader: This is okay in terms of API and Resources, but you can't easily copy and paste it or test it in a browser, because you need to set some header to get the right API version. 3. RequestParam: Here everything is fine. Easy to use, everybody can see, modify, remove the API version.

Basically all three variants have the same implementation: 1. Create an ApiVersion annotation:

@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
public @interface ApiVersion {
    String[] value();
}

2. Extend RequestMappingHandlerMapping and modify getMappingForMethod(...):

@Override
protected RequestMappingInfo getMappingForMethod(Method method, Class<?> handlerType) {
    RequestMappingInfo info = super.getMappingForMethod(method, handlerType);
    ApiVersion methodAnnotation = AnnotationUtils.findAnnotation(method, ApiVersion.class);
    if(methodAnnotation != null) {
        info = requestParamApiVersionInfo(methodAnnotation).combine(info);
        // info = requestHeaderApiVersionInfo(methodAnnotation).combine(info);
        // info = pathVariableApiVersionInfo(methodAnnotation).combine(info);
    } else {
        ApiVersion typeAnnotation = AnnotationUtils.findAnnotation(handlerType, ApiVersion.class);
        if(typeAnnotation != null) {
            info = requestParamApiVersionInfo(typeAnnotation).combine(info);
            // info = requestHeaderApiVersionInfo(typeAnnotation).combine(info);
            // info = pathVariableApiVersionInfo(typeAnnotation).combine(info);
        }
    }
    return info;
}

3. The three requestParam/requestHeader/pathVariableApiVersionInfo methods:

private RequestMappingInfo requestParamApiVersionInfo(ApiVersion annotation) {
    return new RequestMappingInfo(null, null,
        new ParamsRequestCondition(
            Stream.of(annotation.value())
                .map(val -> "v="+val)
                .toArray(String[]::new)
            ),
        null, null, null, null
    );
}

private RequestMappingInfo requestHeaderApiVersionInfo(ApiVersion annotation) {
    return new RequestMappingInfo(null, null, null,
        new HeadersRequestCondition(
            Stream.of(annotation.value())
                .map(val -> "v="+val)
                .toArray(String[]::new)
            ),
        null, null, null
    );
}

private RequestMappingInfo pathVariableApiVersionInfo(ApiVersion annotation) {
    return new RequestMappingInfo(
        new PatternsRequestCondition(
            Stream.of(annotation.value())
                .map(val -> "v"+val)
                .toArray(String[]::new),
            getUrlPathHelper(), getPathMatcher(), useSuffixPatternMatch(), useTrailingSlashMatch(), getFileExtensions()),
        null, null, null, null, null, null
    );
}

I don't know if there's a better way to construct those RequestMappingInfo objects and then combine them, but at least the code works.

This allows things like:

@Controller @RequestMapping(...) @ApiVersion("1.1", "1.2")
public class Foo {
    @RequestMapping(...) @ApiVersion("1.3")
    public void bar() { ... }

    @RequestMapping(...)
    public void baz() { ... }
}

bar() is available only with v=1.3 and baz() can be called via v=1.1 and v=1.2, but not v=1.3. Maybe there should be some parameter, that allows to extend the class level ApiVersion instead of overriding it.

Conclusion: It would be great to have this in Spring (Boot) by default. Of course there are some more advanced use cases, like version ranges and version timestamps, but this should just make it easy for 95% of the people that want to have some simple & convenient Api Versioning within Spring. The last 5% can extend it, or craft something completely different.

[wgorder]

@dsyer I think some blogs might go a long way. Perhaps I posted this prematurely, but it seems every Spring cloud related presentation I attend or watch, API versioning is thrown out there as the solution to the problem I mentioned above. It is always just glazed over and made to sound better than it is I think. In general it creates smells I do not like.

I also think more thought will go into API design going forward as people adjust to working with these new architectures and patterns, but a better way (or documentation thereof) of handling breaking changes to contracts is still needed imo. I don't think I was entirely sure what I was asking for here when I posted this, and perhaps we can close this in lieu of better docs, blogs, and presentations in the future that cover this use case a little better.

HATEOAS is definitely one worth covering (there was a question above that I linked where it seemed to have gone unanswered). Of course here we put more responsibility on those writing the client, and where I work that is not going to be possible for all use cases. I agree with you for those other cases content negotiation seems the most sane.

Both of the strategies leave this history of old API support though that seems really tough to sunset in that deathstar like architecture. One thing I liked about Roy Fieldings post above, is if you were to just throw a new app out there (say v2) they could both run simultaneously, people could upgrade by changing the app name, and eureka could give them the newer one. If the dependency graph gets done that @spencergibb talked about earlier we could find out who is using what to sunset the old versions, and then there is not all of the spaghetti code building up inside your application to support old API's indefinitely on applications with long life spans.

Of course the problem with that approach is you are duplicating entire code bases and now you have 2 to maintain (at least for awhile), you also have additional cost involved with running both versions, at least the way Pivotal CF is currently priced.

[wgorder]

On an aside. I have no problem implementing the 'feature' if I could define it. This post was looking for discussion on if there was anything to implement e.g. a way to support API versioning in a concise maintainable way. Introducing one or 2 changes is not a big deal but with the current tooling over the span of years and many changes I believe it would get difficult to read and maintain, and some applications live a very long time. With no way to sunset old APIs (cant tell who is using what) there would be no way to deprecate or clean up old API's. I guess my real problem as I talk through this is tooling for traceability

[odrotbohm]

While it's generally okay to start a discussion in a ticket, it's usually hard to turn this into actionable tasks, especially if the topic is so broad and controversial.

The number one rule of versioning an API is "don't do it". Most people react by laughing and saying: "Yeah, funny, but I actually need to do it.". My reaction then is: you're still better off considering what you call a "version" of an API what it is: a new API. If the resource identifiers change, it effectively is that.

Also, the reason most of the REST world came to that conclusion is that maintaining multiple "versions" of an API is extremely costly (read here why). This is one of the reasons you'd want to build your API in a away that you don't get to that point where you need breaking changes. Media types, hypermedia all play into that: building servers and clients in a way, that the former can evolve as much as possible without breaking the latter.

That said, I think treating API versioning as a "feature" is an approach that's doomed to fail. You don't simply add an annotation to something and everything works. All the means you need to create loosely coupled, REST-based systems are in place in the Spring stack. I am not arguing each of the parts are perfect already, but the tools are there. Trying to shrinkwrap a certain use case into "a feature" is basically like trying to take a basket of vegetables, meat and potatoes, mash that together, label it "Food" and hope everybody likes it.

[wgorder]

@olivergierke

If there is a better forum for having a discussion to identify (possible) actionable tasks please let me know what it is and I will use it instead.

Just to be clear, there is zero contention from me from everything you said in your last paragraph, I had arrived at much the same conclusion (as I said I perhaps posted prematurely) That said I do have a question on the other bit:

The number one rule of versioning an API is "don't do it". Most people react by laughing and saying: "Yeah, funny, but I actually need to do it.". My reaction then is: you're still better off considering what you call a "version" of an API what it is: a new API. If the resource identifiers change, it effectively is that.

I agree with versioning being something to avoid as I stated above, it also is in agreement with Roy Fielding's opinion (also linked above). I am not so much talking about versioning the API as versioning a resource. Perhaps I am missing something and you can clarify.

A simplistic (and rather unrealistic) example: If I have a Person resource and some developer 2 years ago decided to have an address that encapsulated everything, street, number, city, state, zip in one address field. There are x (unknown) applications out there expecting this address field to contain all this information. We decide that it needs to be broken out into separate fields and we still want 'address' but it wont contain all the extra information rather it will only have the street address.

Now adding the new fields is not a breaking change. Repurposing the address field is a breaking change. Now I realize we could in this case just not repurpose it and call it something else but that is not the point of the example. In this case it is still a person, the expected representation is just now slightly different. If I understand what you are saying we now have to consider this an entirely new API to support this one change. I am ok with that, but how does calling it a new API versus a version change change the problem? What would your solution look like? Again it is a stupid example but I am just trying to follow what you are saying.

[wgorder]

@olivergierke

Or maybe what you are saying is that if the representation of person has to change in a non compatible way that it is a new application at that point? In this case it is kind of inline with my comment above where I said this:

Both of the strategies leave this history of old API support though that seems really tough to sunset in that deathstar like architecture. One thing I liked about Roy Fieldings post above, is if you were to just throw a new app out there (say v2) they could both run simultaneously, people could upgrade by changing the app name, and eureka could give them the newer one. If the dependency graph gets done that @spencergibb talked about earlier we could find out who is using what to sunset the old versions, and then there is not all of the spaghetti code building up inside your application to support old API's indefinitely on applications with long life spans.

Of course the problem with that approach is you are duplicating entire code bases and now you have 2 to maintain (at least for awhile), you also have additional cost involved with running both versions, at least the way Pivotal CF is currently priced.

I will also say that your opinion is not necessarily consistent with recommendations from the Pivotal folks when asked how to address this problem.

[wgorder]

I have come to the conclusion that this whole API versioning thing is a mess and better off avoided, despite the fact that it keeps being offered up as a solution to this problem.

I am leaning towards using the dependency graph currently under development that was mentioned earlier plus some sort of metrics gathering tooling installed at the API gateway layer that can offer insights into API usage. A breaking change if it must be done will just be another application, until the old one can be decommissioned.

I would still be interested in hearing other viewpoints to this if someone wants to post it, but I am going to close this as I don't see a useful feature here to be implemented. I would like there to be more attention given to this common question in the future and hopefully more consistency in the answer as well, but I understand its controversial and I don't think there is a lot of agreement on it.