Add content negotiation to support multiple response formats

[jaumard]

It can be nice to have REST API compatible Protocol Buffers , Flat buffers or Nano buffers in order to save bandwidth.

Here is a video who talk about serialization performance for mobile. And there a graphic about protoBuf and declinaison.

If we want to see more than protobuf we can think of a way to transform API response into another protocol then JSON (eg XML, Protobuf, CSV...). I think this possibility will give to Trails something more than other frameworks.

[tjwebb]

Protocol buffers currently supports generated code in Java, Python, and C++. With our new proto3 language version, you can also work with Go, JavaNano, Ruby, and C#, with more languages to come.

No node.js support? :(

Yea, in general, it would be good to build a trailpack with the ability to transform responses. That's one thing that Sails currently is not very good at. Assuming JSON is the default response, a trailpack could intercept the response and transform it to XML or whatever. Makes sense to me.

[jaumard]

@tjwebb that exactly what I want for Trails ! Sails is not the only one with this limitation. I did notice there no official implementation for node.js yet but there some alternative implementations, here is some : https://www.npmjs.com/package/protocol-buffers https://github.com/dcodeIO/ProtoBuf.js

[tjwebb]

Yea, I think many frameworks don't have this capability, not just Sails :)

I know Grails can support at least json and xml, but I'm not aware of a plugin system that would allow many response formats. This would be pretty cool. The API could be defined in a trailpack that does content negotiation. I think this is a pretty good resource on how it works in Grails: http://grails.github.io/grails-doc/2.3.x/guide/single.html#contentNegotiation

[jaumard]

Look good to do contentNegotiation with Accept or ?format=xml but how integrate this info Trails ? Do you think a trailpack-contentnegotiation can do this job ?

The only way I see how implement this is to allow some hooks under web server trailpacks to allow footprints transformation before sending the data. The problem I see it's user controller can't use them easily...

Or add a method sendContent (or whatever) in trails-controller that can call hook from trailpack-contentnegotiation. Like this it can have multiple trailpacks that can transform data and any controller can call them.

What do you think ?

[jaumard]

No node.js support? :(

Maybe there website are not up to date because on the repo I see there some JavaScript support : https://github.com/google/protobuf/tree/master/js

[jaumard]

Here is my proposal for this functionalitiy :

We can use the config/trailpack.js to have something like this (for trailpack hapi/express...):

...
hooks: ['formatContent', 'parseContent'], 
....
provides: {},

Now express can have a middleware (or hapi plugins) to parse and format datas into supported formats. Or trails-controller can have an utile methods to parse/format and each controllers can decide to use it or not...

Under a trailpacks who want to implement this hook :

provides: {
    ...
    hooks: {
       'formatContent': 'Hook.formatContent',
       'parseContent': 'Hook.parseContent'
    } 
    ....
  },

File is under api/hooks/Hook.js :

module.exports = class Hook {
    parseContent(bodyToParse, next) {//and/or Promise
            //parse and return the bodyToParse param into JSON 
            //this is useful when server receive data in XML, Protobuf....
    },
    formatContent(bodyToFormat, format, next) {//and/or Promise
            //Format to _format_ if supported and return the bodyToFormat
    }
}

With this methods Trailpacks become more powerful because they can now interact with each others not only for contentNegotiation...

[jaumard]

Little up here :) @trailsjs/maintainers @tjwebb any feedback on this ? I just think on another way to do it, maybe we can use a simple middleware that will be executed after Footprints controller. Like this it will be very simple to implement and use.

First footprints controller doesn't return the values directly, it put it under req to pass it into the next middleware.

Then we have a default middleware that does this res.reply or res.json.

Users or Trailpack can override this middleware in order to change output format they want.