search

elixir-lang / gen_stage

Producer and consumer actors with back-pressure for Elixir
http://hexdocs.pm/gen_stage
1.52k stars 193 forks source link

GenStage.init should be consistent with GenServer.init and support timeout and :hibernate #103

Closed almightycouch closed 8 years ago

almightycouch commented 8 years ago

GenServer.init/1 supports a third parameter to timeout or hibernate the process. For consistency reason, GenStage.init/1 should take an optional fourth parameter for this purpose.

josevalim commented 8 years ago

So... it is hard to support timeouts and hibernate because it receives multiple messages on the side, completely transparent from the user. So the only way to emulate timeouts is by setting timers, so the user can do it themselves. Hibernation would need to be done with an attribute on the state, so we go back to hibernate after receiving stage bookeeping messages.

fishcakez commented 8 years ago

Just a note for anyone writing their own behaviour. Using timers for OTP behaviour callbacks is considered best practise by the OTP team. GenServer uses receive do .. after .. end and is only kept for backwards compatibility, so even if you are building on top of GenServer and not handling "extra" messages you will want to consider using timers if you want accurate timeouts. Of course if not doing any introspection it should not be an issue.

Using receive after has the same issue as GenStage would experience here because the timeout would get restarted when a system message is received. If using :observer, which can poll a process to introspect its state using system messages, or similar tool, its possible to repeatedly delay the timeout.

almightycouch commented 8 years ago

Oh, I was almost sure that GenStage would rely on the same semantics than GenServer. It doesn't.

Implementing timeouts using Process.send_after/3 would be a possible solution. But the user could do the same in init/1 anyway...

so we go back to hibernate after receiving stage bookeeping messages.

In this case, I'm not sure if hibernate makes a lot of sense, maybe for very exotic scenarios. I was not aware that GenStage was processing messages after the init function. Just for curiosity, what do you mean by bookeeping ?

GenServer uses receive do .. after .. end and is only kept for backwards compatibility.

Thank you for the infos, I was wondering if there are any differences for using Process.send_after/3 instead of using the third argument in init/1.

fishcakez commented 8 years ago

@almightycouch "book keeping" means ask/subscribe/cancel/event etc messages and monitor signals from subscribed processes. These messages set up flow control between stages and other things.

:erlang.start_timer is probably a better fit than Process.send_after/3 for this use case because there is a single reference to track, which makes it easier to identify, cancel and flush the timer when required. This is very important if there is a callback module because don't want to leak the message. I am unsure of a situation where I would use Process.send_after/3.