Automatic batching of action requests

[GoogleCodeExporter]

In Google I/O 2010 - Architecting for performance with GWT, they discuss having 
too many rpc request and automatically batching of actions. Different parts of 
the application need to request information from the server, and these should 
be automatically batched together and sent over one rpc request.

http://code.google.com/events/io/2010/sessions/architecting-performance-gwt.html
19:54 The other issue, as I mentioned before, is the http requests.
19:59 So, this is a super-simplified version or imaginative version of the Wave 
Client server
20:05 protocol. It doesn't actually work this way for reasons
20:08 that will become clear in a moment. Now, you might say, "Well, I'll just 
have
20:15 each piece of my UI talk to an RPC interface, get data that it needs from 
the back end."
20:20 This is written in terms of the GWT RPC semantics roughly, but the same 
thing applies no matter
20:26 what you're using: Json, XML, or what have you.
20:29 The problem with this is that, very often, a single user interaction 
leads to a bunch
20:34 of requests. And the reason this happens is that you say,
20:37 imagine I'm in wave, I click on a wave. I've got to fetch the header, to 
display that.
20:44 I've got to fetch the wave itself, so I can display that.
20:47 But also, because I want to keep the rest of the UI up to date, I use 
this opportunity
20:53 also go fetch the status of my contacts. I have to fetch the inbox, an 
updated version
20:59 of the inbox perhaps, that's deltas, or what have you.
21:02 You get the idea. I'm fetching a bunch of stuff from one user
21:05 interaction. Well, this would translate into four http requests, and they 
can get serialized
21:10 because mini-browsers have a 2-connection limit.
21:13 Even the modern browsers will have a 2-connection limit on the slow 
connection so they don't
21:18 saturate the link. So that ends up being really, really bad in
21:20 practice when your UI has, it really has two problems, one, it's slow.
21:24 Two, it feels really slow, because these things come in at different 
rates.
21:27 So, this pops in, that pops in, that pops in, that pops in.
21:30 The whole UI kind of jiggles and feels awkward and sluggish.
21:33 The easy way to fix this, well, relatively easy, is to actually 
restructure your RPC
21:40 interface such that you break it into "request object" and "response 
objects."
21:46 You build a single interface, at least for the purpose of most of the UI, 
that allows
21:50 you to batch all of those requests. So, all the different parts of the UI 
that
21:54 need information will populate an array of request objects, array of list 
request objects,
22:00 go to the server, server processes all those, sends back the response 
objects, and the piece
22:05 of code doing the transfer goes off and farms those responses off of the 
appropriate parts
22:10 of the UI. And a really simple way of doing this is simply
22:15 to use a deferred command. If you're more familiar with JavaScript, it's
22:18 equivalent to using "set timeout zero," more or less.
22:21 And that allows you to aggregate your requests based on a single user 
interaction.
22:26 The idea is that a deferred command will run after the current event 
handler is done, which
22:30 almost invariably corresponds to a single user action.
22:33 And when that thing fires, you take all the requests that have been 
batched up by different
22:37 parts of the UI, and it fires them off to the server to be handled.
22:41 That allows the server to optimize things, so like hot database 
connections and so forth
22:44 can be used for all these different requests being made.
22:48 And it allows the different parts of the UI to be written independently, 
because they
22:51 all just use this simple add request interface to get their data.
22:55 They don't need to know about each other, which keeps your code simpler.
23:00 And that can lead to, again, a single http request for user action, which 
is your goal.

Original issue reported on code.google.com by brendanp...@gmail.com on 11 Jul 2010 at 10:36

[GoogleCodeExporter]

This is actually something undocummented and that will need to be reviewed. 
Batching is currently supported.

http://code.google.com/docreader/#p=gwt-dispatch&s=gwt-dispatch&t=CompoundAction
s

Almost nothing have changed from this.

Original comment by goudreau...@gmail.com on 11 Jul 2010 at 3:50

[GoogleCodeExporter]

I'm familiar with BatchActions, but what I think what they are talking about is 
automatic batching of actions.  Different parts of the UI (that don't know 
about each other) all make requests, but instead of making the request they are 
put in a queue a DeferredCommand is used to "take all the requests that have 
been batched up by different parts of the UI, and it fires them off to the 
server to be handled."

BatchCommand requires the developer to batch actions together.  What I'm 
suggesting is that gwt-platform does the automatic batching of requests. 

Original comment by brendanp...@gmail.com on 11 Jul 2010 at 7:46

[GoogleCodeExporter]

Here is a quick hack I made to DefaultDispatchAsync. I hope it shows roughly 
what I'm intending to achieve.  

The code doesn't compile, as it has a 'Bound Mismatch' problem.

Original comment by brendanp...@gmail.com on 11 Jul 2010 at 8:35

Attachments:

• autobatching_hack.patch

[GoogleCodeExporter]

I checked your patch, I like the idea, but it does get rid of the principle 
that actions can have different entry point URLs. So:
1) Do we try to reconcile both features? (i.e. we could keep a different queue 
for each  entry point URL)
2) Do we drop support for different entry point URLs?
3) Do we make auto-batching incompatible with different entry point URLs?

Also, I haven't tried applying it, but what is causing this 'Bound Mismatch' 
problem?

Original comment by philippe.beaudoin on 12 Jul 2010 at 7:02

• Added labels: Milestone-V1.0-Release

[GoogleCodeExporter]

1) I don't know, I really like the way rpc calls are made. Batching manually 
when I can.
2) No.
3) If we do this, we could make something up that will create a batching action 
sting like URL.

4) Queueing is something that we should support. If not automatic, at least 
manually. Something like : 
dispatcher.addToQueue(new action, new asynccallback);
dispatcher.commitQueue().

I have to say that I have no idea if we can make the assumption that the 
automatic queue will be called when I want as a developer and I have some cases 
where it would be awesome to be able to control when and how it works.

Original comment by goudreau...@gmail.com on 12 Jul 2010 at 12:20

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Did you forget to bind BatchActionHandler ?

Original comment by goudreau...@gmail.com on 12 Jul 2010 at 12:44

[GoogleCodeExporter]

I have never run into a situation requiring autobatching, but I can imagine 
some cases where it's usefule. Brendan, can you give us an example where it 
would have made your life easier? 

I don't think autobatching would produce any behavior change from the point of 
view of the user -- beside a performance/memory hit. As such, I'm not a big fan 
of adding methods to the API as proposed by Christian in (4). Maybe a global 
switch to turn  autobatching on/off on a specific entry point URL?

My proposition for now:
- Do autobatching in a subclass (i.e. AutoBatchingDispatchAsync)
- Let the user bind the default or autobatching version (i.e. if he wants to 
save memory)
- When using AutoBatching, the user has to explicitely enable autobatching on 
specific entry point urls, say via a call to 
enableAutoBatching("/my/entrypoint/url")
- AutoBatchingDispatchAsync instantiates 1 queue for each enabled entry point 
URL.

The only thing I'm not sure about is the 1st step (I'm not even sure there is a 
real performance/memory cost due to GWT's optimized compilation).

Original comment by philippe.beaudoin on 12 Jul 2010 at 5:22

[GoogleCodeExporter]

Forgot to add:
- If an action is fired with an entry point url that is not autobatched, it is 
sent right away to the server.

Original comment by philippe.beaudoin on 12 Jul 2010 at 5:23

[GoogleCodeExporter]

/Maybe a global switch to turn  autobatching on/off on a specific entry point 
URL?
Just one clarification, I was talking about manually Queuing, when you need an 
action to be done and have the result before firing the next one. And for that, 
yes a simple override with a true/false statement would be awesome.

Batching could be done in the same way. We could even bind the default boolean 
values with gin instead of having to add a parameter to every execute.

Original comment by goudreau...@gmail.com on 12 Jul 2010 at 5:39

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

I don't have a personal example, but listened to the Google I/O sessions and 
thought we can learn from Google's experience.  If you listen to the Google I/O 
session I linked to, the aussie guy said he needed it to fix performance issues 
in Google Wave.  Say you were rewriting gmail, on application startup the inbox 
presenter will want to grab the first page of the inbox, and the menu presenter 
will want to grab the total unread emails.  Each of these presenters can avoid 
knowing about each other.  Any yet only make one http connection.

I think autobatching is a performance enhancement (to avoid hitting the limit 
on http connections - mobile browser have a limit of 2, old browser limit of 
4), and it would probably make it harder to debug.  So we don't want to force 
it on everyone.  

I see 3 switches to choose from:
- global, either all on or all off.
- per action, each action defines an boolean autoBatch() function.
- per call, dispatcher.executeDelayed(...)

The reason for autobatching instead of manual batching is reduced coupling.  
One part of the application doesn't have to know that another part of the 
application is requesting information.   

My patch had issues mainly as I only had a short time to code it, and I just 
wanted to get the concept out for discussion.  I don't expect any of the code 
to be used anyway.

Original comment by brendanp...@gmail.com on 12 Jul 2010 at 8:03

[GoogleCodeExporter]

Yeah and in the cases you need gwt-comet on one connection, you have one 
connection left for those poor little mobile browsers :)

Original comment by goudreau...@gmail.com on 12 Jul 2010 at 8:03

[GoogleCodeExporter]

such a dispatcher should support many strategies, like commiting, as mentioned 
above or my favourite would be auto-sensing, where the dispatcher waits for a 
certain period of time after last dispatch instruction which is queued but not 
executed. after certain timeout of being idle, dispatcher would transmit them 
as a batch and proceed callback execution

Original comment by eplisc...@gmail.com on 18 Aug 2010 at 10:08

[GoogleCodeExporter]

Original comment by philippe.beaudoin on 22 Sep 2010 at 1:40

• Added labels: Component-Dispatch

[GoogleCodeExporter]

Original comment by bren...@doherty.net.nz on 6 Oct 2010 at 9:52