API endpoint concatenation

[sirkitree]

James and I were brainstorming about decoupled approaches last night, and James asked how we could keep down requests on a page when there are things like the main content, and various sidebar blocks, etc on any given page. Ideally each of those areas is going to have it's own API endpoint which returns the relevant data/content from Drupal. So each section would typically be a new call to the API in order to pull the various content pieces together for a page.

In order to reduce the amount of calls on a given page, it would be pretty awesome to be able to concatenate or create a facade endpoint for the various endpoints you want to call for a given page. For example:

Let's say I have three endpoints:

• GET/main-content?param1=foo
• GET/sidebar1-content?param1=this&param2=that
• GET/sidebar2-content

In the case where we have something like an AngularJS front end, we'd create a service that calls each of these endpoints individually, having three calls, and three returns.

Now if we had a facade endpoint or pipe endpoint which we could make one request and say that we need each of these at the same time, and retrieve the data for each in one payload, that would reduce the amount of calls you have to make. Something like:

• GET/pipeline?main-content=param1:foo&sidebar1-content=param1:this,param2:that&sidebar2-content

(not entirely sure that is valid format but you get the idea, right?)

Then the payload returned from this call would be your standard json object and each response would be structured accordingly to the endpoints involved.

I'm not entirely sure how this would fit into createapi itself, but I wanted to throw out the idea in case you think it could be something that would make sense to be within createapi and could provide some advice on where I might start or if you even think it makes sense at all.

[justafish]

I think this is mixing the concepts of data and presentation by thinking of it as a "sidebar feed", in fact you can already achieve this.

A good example is the LLX Site, which has "Touts". These are just content types with a title and a blob of text (so like a block). Different positions in the front-end app can be filled by touts/blocks, which are configured in the back end:

The initial site settings payload then gives you the IDs for these: http://cms.londonlindyexchange.com/api/1.0/site-information.json

Which you can then get more information out of from the touts feed: http://cms.londonlindyexchange.com/api/1.0/touts.json

[justafish]

See also:

https://github.com/justafish/llx/blob/master/angular/app/js/controllers.js#L95 https://github.com/justafish/llx/blob/master/angular/app/_partials/global_sidebar.html

[sirkitree]

I'm not really talking about the sidebar, that was just an example to illustrate that there are different areas on the site that would require multiple calls to the API. The idea is that if you have a lot of API calls on one page, it would be able to still get all of the data needed for a page through one call instead of multiple calls.

And if that is possible, I thought it might be a god generalized function within this module to provide to it's users.

The examples you're giving here seem like you're still making two calls vs one anyway (if i'm reading this correctly).

[justafish]

Ahh I see. Do you know roughly how many individual feeds would need concatenating for this to become a benefit? Like, what's the overhead for making a request and how many concurrent requests can you do at one time? Since instead of being able to process highly cacheable individual feeds and render them as they arrive, now the page is entirely held up by one large data dump coming from a source that potentially needs building when requested.

[sirkitree]

Do you know roughly how many individual feeds would need concatenating for this to become a benefit? Nope, it's all theoretical at this point, hence just discussing the merits. I'm sure there are tradeoffs and those tradeoffs would be project to project (one project might want to do a request for three things at once, and another call separately, while another project might want everything). Personally I would go by the 'page fold', trying to do one call for everything above the fold in an initial call and then lazy load others on down the page. (cc: @q0rban to see if he has any thoughts here)

[q0rban]

I was searching around for combining multiple AJAX requests into one, and came across this.

http://my.safaribooksonline.com/book/web-development/microsoft-aspdotnet/9780596510503/optimizing-aspdotnet-ajax/combining_multiple_ajax_calls_into_one_c

Network roundtrip is the most expensive part of Ajax applications. You need to do everything you can to reduce the number of calls made to the server. Because each call has the overhead of connecting to the server and then downloading the response, each call wastes some time on the network. When you have five calls going to the server within a short time for some particular operation, you are wasting about one second (assuming 200 ms network latency) on the network. So, a popular practice among Ajax developers is to batch multiple consecutive single calls into one large call. This saves the network roundtrip as there's only one network roundtrip to do, and thus the whole operation executes faster than making individual calls.