search

djpate / jflow

jRuby compatible library for AWS SWF
2 stars 3 forks source link

Poll Spams AWS SWF API #9

Open dayyan opened 7 years ago

dayyan commented 7 years ago

Looks like the polling mechanism polls like crazy. 

I, [2016-11-18T23:30:30.450000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:30.462000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:30.470000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:30.482000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:30.503000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:30.512000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:30.528000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:30.843000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:31.156000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:31.477000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:31.788000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:32.103000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:32.414000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:32.725000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:34.844000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:36.967000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:39.090000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:39.099000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:41.221000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:43.364000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:43.694000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***
I, [2016-11-18T23:30:45.812000 #4545]  INFO -- : [4972] Polling for task on sec-flow-*** - ***_activity_tasklist_***

The worker should allow a wait period to be specified, with a default. I'm guessing it would be a good idea to introduce that somewhere here: https://github.com/djpate/jflow/blob/master/lib/jflow/activity/worker.rb#L25-L52

djpate commented 7 years ago

Polling from AWS takes about 60 secs IIRC. This might be each worker polling?

JFlow.configuration.swf_client.poll_for_activity_task(poll_params)

This should block for 60sec

dayyan commented 7 years ago

I think the [4972] above is the same worker, is it? https://github.com/djpate/jflow/blob/master/lib/jflow/activity/worker.rb#L61