bipio-server / bipio

The bip.io API Server
https://bip.io
Apache License 2.0
873 stars 117 forks source link

Continue the process, after recovery the broken bipio server. #46

Closed EricCat closed 9 years ago

EricCat commented 9 years ago

For example, there is a bip with three channels, when channel 2 processing the workflow, bipio server breaking down. After restart bipio server, I want to process the workflow from channel 2, not start from beginning. How to deal with this situation? I think maybe need to handle rabbitmq queue. Is there some api for handle rabbitmq ?

mjpearson commented 9 years ago

Are you seeing a specific error being raised? The number of channels is arbitrary, but if a specific error is on a pod then investigation can be focused. Can you please post any output received by the server (if any?) or otherwise post your transforms configuration (minus any personal info)

EricCat commented 9 years ago

@mjackson Thank you. Sorry, maybe I'm not explain clear. This is not a error, just a feature I want to implement. For example: I build the bip Web Hook Template To FB And Twitter: When the workflow processing on twitter.status_update (not finishing), the bipio server shuts down(e.g. killed by OS for memory usage, uncaught exceptionm, etc.). After bipio server restart, I want to go on the workflow from twitter.status_update, not from template.text_template. In short, when the bipio server is restarted , if the data can be restored in the rabbitmq queue? How bipio handle this situation?

Some ref maybe could explain more clear
mjpearson commented 9 years ago

Hi, thanks for clarifying :)

durability can be tweaked by in the server config (config/default.json) - the rabbit > exchanges > bastion_generic > queue_default which are the node-amqp constructor options for setting durability on the default job queue (defaults to on).

processing should continue if either the bipio server or rabbitmq server is restarted, as the payloads in rabbit always contain their parent graph and a pointer to the next hop. If you're finding that processing isn't continuing after a reboot then that's probably a bug and needs investigating

EricCat commented 9 years ago

@mjackson Thanks for your explain.