inaka / worker_pool

Erlang worker pool
https://hex.pm/packages/worker_pool
Apache License 2.0
276 stars 80 forks source link

[Feature] Question about possible feature #125

Closed JrIndie closed 7 years ago

JrIndie commented 7 years ago

I was reading the documentation and I was wondering if it would be possible to make sure a worker can not start before another worker finishes their work. What I mean by this is ... if you have a gen_tcp:recv and you receive some data , and in that data are two commands (based on whatever protocol you are using). Not an issue, but if you process the data, Command A, and then send it off to a worker, and then process Command B, and send it off to a worker -- there is a real world chance Command B finishes before Command A due to whatever issues that may happen. Maybe Command A is a process intensive, maybe Command A gets blocked for whatever reason, etc, etc.

Anyway, does worker_pool support a way to make sure all calls from a PID happen in sequence? As you would not want to process Command B unless Command A was finished. Or should you just handle this in the gen_tcp:recv logic?

Thanks!

elbrujohalcon commented 7 years ago

@JrIndie why don't you sequence the calls yourself? Using wpool:call(…) for both Command A and Command B?

elbrujohalcon commented 7 years ago

Or maybe a single wpool:cast(your_pool, [CommandA, CommandB]) so that your worker can sequence the commands internally on its own?

JrIndie commented 7 years ago

@elbrujohalcon For 1: Assuming Command B is missing 1 byte, and the next gen_tcp:recv completes the data, it will then be processed. For 2: There could be 100 Commands combined on the buffer.

I guess I could just make sure if I wpool:call, that I check for a response before pushing another wpool:call if the Command needs to be in order?

elbrujohalcon commented 7 years ago

@JrIndie yeah… In general I would not push the responsibility of sequencing commands to the workers themselves. The sequencing of commands should be a responsibility of the caller.