search

dashbitco / broadway

Concurrent and multi-stage data ingestion and data processing with Elixir
https://elixir-broadway.org
Apache License 2.0
2.43k stars 161 forks source link

Terminator reporting error with intentional shutdown #89

Closed fcevado closed 5 years ago

fcevado commented 5 years ago

I have a custom producer that consumes from an api so my producer achieve a point where there is no events to produce anymore and when all events are successfully acknowledged my broadway tree can go down, when this happens i send a message to stop the Broadway GenServer that is the beginning of the tree. i do that using GenServert.stop/2. My producer implements GenStage, Acknowledger and Producer behaviours. It handles ack/3, prepare_for_draining/1 and terminate/2 callbacks. When I send the message to the Broadway GenServer it logs this error message:

[error] Supervisor '<MyBroadway>.Supervisor' had child '<MyBroadway>.Terminator' started with 'Elixir.Broadway.Terminator':start_link([{producers,['<MyBroadway>.Producer_default_1']},{first,['<MyBroadway>.Processor_default_1',...]},...], [{name,'<MyBroadway>.Terminator'}]) at <0.532.0> exit with reason killed in context shutdown_error

Should i cover something else to avoid the error message? There is any config that i can pass to Broadway.start_link that gonna make the tree temporary? or this is a expected behavior?

josevalim commented 5 years ago

I assume what you have is a deadlock. GenServer.stop/2 is synchronous, so if you are calling that from any process inside Broadway itself, the terminator will wait for that process to terminate but it won't, because that process is waiting on the tree itself to terminate. We would need to introduce a Broadway.stop and Broadway.async_stop APIs for you to do what you want.

fcevado commented 5 years ago

Thanks @josevalim, you're right. I can move the GenServer.stop/2 call to an external process.