search

GsDevKit / ServiceVM

The Service VM is intended to provide example code for creating and using a separate "Service VM" for offloading work that in a Squeak/Pharo Seaside application, you would have forked of a thread to do the work
MIT License
2 stars 3 forks source link

service vm needs better support for handling commit conflicts #3

Open dalehenrich opened 10 years ago

dalehenrich commented 10 years ago

https://code.google.com/p/glassdb/issues/detail?id=326

dalehenrich commented 10 years ago

@jbrichau, was this covered by https://code.google.com/p/glassdb/issues/detail?id=355

jbrichau commented 10 years ago

@dalehenrich I think that issue was more about 'what if commit conflicts happen, how can we handle them inside the service vm'. I think you created the issue because I said it was quite hard to discover what was going on inside the service vm. I'm not sure what can be done about that and if it is even necessary?

dalehenrich commented 10 years ago

Okay that sounds fine ... so better support is stiil needed ... BTW, you are using service vms, right? could you share some information about what you are doing?

Obviously I'm interested in blowing the dust of this (and other projects) but I don't want to break what you are doing ... are you using Nick's futures work or are you using the basic service ...

There are several paths ... one is to just get everything pulled together in one place with proper updated scripts (and an install script) ... a nice working example using Zinc and the tODE scripts needed to setup/configurat the vm, launch service vm, launch zinc and run the example .... in a nice neat package so if you have fixes etc. that are easily sharable (or if you just threaten to share, I will wait for you since I don't want to duplicate effort or head off in wildly different directions ...)

the other is to take the futures work from Nick and bring it into the mix

the third is to take some of the Promises done in squeak by Frank Shearar and hook it up ...

At this moment I am not an expert on Futures/Promises ... I'm just a plumber hooking things up and patching the walls, but we could push into new ground if that is called fo...

Dale

On Fri, Jun 6, 2014 at 11:32 AM, Johan Brichau notifications@github.com wrote:

@dalehenrich https://github.com/dalehenrich I think that issue was more about 'what if commit conflicts happen, how can we handle them inside the service vm'. I think you created the issue because I said it was quite hard to discover what was going on inside the service vm. I'm not sure what can be done about that and if it is even necessary?

— Reply to this email directly or view it on GitHub https://github.com/glassdb/ServiceVM/issues/3#issuecomment-45369871.

jbrichau commented 10 years ago

We're using the basic service (i.e. the packages included with Seaside in GemStone) but have written an integration with our DALi framework that pretty much cross-cuts the code. Never used the Futures work either.

Our use is straightforward: in a (Seaside) request, we fork off a task to execute, which we store in the database and can poll them regularly when/if they are finished. Something we found useful is a separation between the code that should run inside a transaction and the code that should run outside of the transaction. After fixing the bug with doTransaction, the service vm basically executes the tasks in sequence (because they each should run in their own transaction). We improved that by splitting a task into two blocks to execute. The first one executes outside of the transaction and is intended for code that does not have anything to commit (so when other tasks commit their transaction, it's never made persistent). The results of the block is then passed to the second block, which runs inside a transaction and does the persistent work.

We do have a bugfix for the serviceVMTaskServiceExample which we should share. I should dig up what it exactly fixes but I think the servicevm got stuck when more than 100 tasks were queued.v It was not published because it dated before we fixed the doTransaction bug and we originally thought that was the bug that blocked our vms. Since we continued seeing it, we thought we did not fix anything, and eventually hit the fix for doTransaction. But I'll dig up that bug in this context and get the fix merged in here, for sure.

dalehenrich commented 10 years ago

Thanks very much ... this is very helpful and I want to make sure that I don't collide interfere with your work ... I like the two-block idea separation is real good and not having to explicitly write transaction code is a very good thing ...

as with all of my projects I'm sure the work will go slowly, but I've always thought that the server vm was a valuable thing to have and I've always thought that the server vm did receive enough love from the documentation aspect and that I will clean up in the short term ...