Killing Genserver using persistent_ets kills my entire application

[amacciola]

I have a Genserver that is using persistent_ets as a persistent cache. I also have a Supervisor running that is set to restart the Genserver on any types of failures. However if i manually kill the process hosting the Genserver with persistent_ets then it will print out this message:

iex(3)>Process.exit(pid_1, :kill) iex(4)>[error] GenServer #PID<0.635.0> terminating ** (stop) killed Last message: {:"ETS-TRANSFER", :event_consumer_cache_table, #PID<0.634.0>, :inherited} State: %PersistentEts.TableManager{keypos: 1, path: 'priv/ets_persisted_data/event_consumer_cache_table.tab', persist_every: 60000, persist_opts: [], protection: :protected, table: :event_consumer_cache_table, table_opts: [:named_table], timer: #Reference<0.883420884.3413114881.16364>, type: :set}

and then it will crash my entire phoenix application. Is there something i am missing in dealing with crashes and persistent_ets ?

Thank You

[michalmuskala]

Do you have a way to reproduce this? It's not clear to me from the description alone what is happening.

[amacciola]

@michalmuskala yes i can recreate this easily and i can try to explain the situation in more detail. Basically i have a Genserver that is using ETS as an in memory cache. Then on top of that i am using this persistent ets library to persist data to the filesystem. Now when i have my Genserver running and i grab the PID of its process and kill it i get this error.

iex(4)>[error] GenServer #PID<0.635.0> terminating ** (stop) killed Last message: {:"ETS-TRANSFER", :event_consumer_cache_table, #PID<0.634.0>, :inherited} State: %PersistentEts.TableManager{keypos: 1, path: 'priv/ets_persisted_data/event_consumer_cache_table.tab', persist_every: 60000, persist_opts: [], protection: :protected, table: :event_consumer_cache_table, table_opts: [:named_table], timer: #Reference<0.883420884.3413114881.16364>, type: :set}

and then my entire application exits. Where instead my Supervisor should just have restarted my Genserver since it has crashed. The init() method of my Genserver looks like this

@impl true
  def init(args) do
    [{:ets_table_name, ets_table_name}, {:log_limit, log_limit}] = args
    PersistentEts.new(ets_table_name, [:named_table, :set])
    {:ok, %{log_limit: log_limit, ets_table_name: ets_table_name}}
  end

Now if i change this to instead just use :ets.new(ets_table_name, [:named_table, :set])

my application has no issues restarting the Genserver once i kill it.

I hope this explanation helps.

[amacciola]

@michalmuskala any update on this issue ? or anything else you need to be provided ?

[michalmuskala]

This is in my backlog, I'm not sure exactly when I'll be able to get to it, hopefully either this week or the weekend.

It would help tremendously if I had a reproduction case I could clone and run to see the issue.

[optikfluffel]

I stumbled upon the same problem yesterday and tried to setup a minimal example at optikfluffel/persistent_ets_bug. I hope this helps solving the issue.

[EliriaT]

Hi. I have a similar issue. Are there maybe any possible updates on how to solve this problem? In my case, I have a Genserver that is using the persistent_ets. In case the Genserver is killed, the supervisor fails to restart the killed Genserver. Otherwise, without using the persistent_ets, everything behaves as expected and the Genserver is restarted by its supervisor.

Also, for some strange reason, if I create the same Genserver with the same persistent_ets a second time after it was previously killed, everything seems to work properly, and the Genserver is restarted by its supervisor.