boundary / folsom

Expose Erlang Events and Metrics
Apache License 2.0
586 stars 166 forks source link

A race condition can stop trimming of slide servers #100

Open solvip opened 8 years ago

solvip commented 8 years ago

If SampleMod:trim/2 gets called by a folsom_sample_slide_server after the ETS table the server is trimming has gone away, it will go into a restart loop and eventually bring down the folsom_sample_slide_sup supervisor.

Pull request #99 fixes this.

A log section that shows this happening:

2015-10-27 12:40:03.889 [info] <0.15943.9>@ws_handler:websocket_terminate:278 Authenticated client(...) from ... disconnected
2015-10-27 12:40:03.889 [error] <0.15942.9> gen_server <0.15942.9> terminated with reason: bad argument in call to ets:select_delete(2230132366, [{{{'$1','_'},'_'},[{'<','$1',1445949598}],[true]}]) in folsom_sample_slide:trim/2 line 64
2015-10-27 12:40:03.890 [error] <0.15942.9> CRASH REPORT Process <0.15942.9> with 0 neighbours exited with reason: bad argument in call to ets:select_delete(2230132366, [{{{'$1','_'},'_'},[{'<','$1',1445949598}],[true]}]) in folsom_sample_slide:trim/2 line 64
2015-10-27 12:40:03.890 [error] <0.7667.3> Supervisor folsom_sample_slide_sup had child undefined started with folsom_sample_slide_server:start_link(folsom_sample_slide, 2230132366, 5) at <0.15942.9> exit with reason bad argument in call to ets:select_delete(2230132366, [{{{'$1','_'},'_'},[{'<','$1',1445949598}],[true]}]) in folsom_sample_slide:trim/2 line 64 in context child_terminated
2015-10-27 12:40:06.394 [error] <0.16357.9> gen_server <0.16357.9> terminated with reason: bad argument in call to ets:select_delete(2230132366, [{{{'$1','_'},'_'},[{'<','$1',1445949601}],[true]}]) in folsom_sample_slide:trim/2 line 64
2015-10-27 12:40:06.394 [error] <0.16357.9> CRASH REPORT Process <0.16357.9> with 0 neighbours exited with reason: bad argument in call to ets:select_delete(2230132366, [{{{'$1','_'},'_'},[{'<','$1',1445949601}],[true]}]) in folsom_sample_slide:trim/2 line 64
2015-10-27 12:40:06.394 [error] <0.7667.3> Supervisor folsom_sample_slide_sup had child undefined started with folsom_sample_slide_server:start_link(folsom_sample_slide, 2230132366, 5) at <0.16357.9> exit with reason bad argument in call to ets:select_delete(2230132366, [{{{'$1','_'},'_'},[{'<','$1',1445949601}],[true]}]) in folsom_sample_slide:trim/2 line 64 in context child_terminated
2015-10-27 12:40:08.895 [error] <0.16365.9> gen_server <0.16365.9> terminated with reason: bad argument in call to ets:select_delete(2230132366, [{{{'$1','_'},'_'},[{'<','$1',1445949603}],[true]}]) in folsom_sample_slide:trim/2 line 64
2015-10-27 12:40:08.895 [error] <0.16365.9> CRASH REPORT Process <0.16365.9> with 0 neighbours exited with reason: bad argument in call to ets:select_delete(2230132366, [{{{'$1','_'},'_'},[{'<','$1',1445949603}],[true]}]) in folsom_sample_slide:trim/2 line 64
2015-10-27 12:40:08.895 [error] <0.7667.3> Supervisor folsom_sample_slide_sup had child undefined started with folsom_sample_slide_server:start_link(folsom_sample_slide, 2230132366, 5) at <0.16365.9> exit with reason bad argument in call to ets:select_delete(2230132366, [{{{'$1','_'},'_'},[{'<','$1',1445949603}],[true]}]) in folsom_sample_slide:trim/2 line 64 in context child_terminated
2015-10-27 12:40:11.396 [error] <0.16368.9> gen_server <0.16368.9> terminated with reason: bad argument in call to ets:select_delete(2230132366, [{{{'$1','_'},'_'},[{'<','$1',1445949606}],[true]}]) in folsom_sample_slide:trim/2 line 64
2015-10-27 12:40:11.396 [error] <0.16368.9> CRASH REPORT Process <0.16368.9> with 0 neighbours exited with reason: bad argument in call to ets:select_delete(2230132366, [{{{'$1','_'},'_'},[{'<','$1',1445949606}],[true]}]) in folsom_sample_slide:trim/2 line 64
2015-10-27 12:40:11.396 [error] <0.7667.3> Supervisor folsom_sample_slide_sup had child undefined started with folsom_sample_slide_server:start_link(folsom_sample_slide, 2230132366, 5) at <0.16368.9> exit with reason bad argument in call to ets:select_delete(2230132366, [{{{'$1','_'},'_'},[{'<','$1',1445949606}],[true]}]) in folsom_sample_slide:trim/2 line 64 in context child_terminated
2015-10-27 12:40:11.397 [error] <0.7667.3> Supervisor folsom_sample_slide_sup had child undefined started with folsom_sample_slide_server:start_link(folsom_sample_slide, 2230132366, 5) at <0.16368.9> exit with reason reached_max_restart_intensity in context shutdown
2015-10-27 12:40:11.397 [error] <0.485.0> Supervisor folsom_sup had child folsom_sample_slide_sup started with folsom_sample_slide_sup:start_link() at <0.7667.3> exit with reason shutdown in context child_terminated
joewilliams commented 8 years ago

Folsom has moved, please resubmit your issue at https://github.com/folsom-project Thanks!

jfacorro commented 8 years ago

@joewilliams You might already know this, but you can transfer the repository to another owner, that way the project won't loose all its related information, like all the issues.