whatyouhide / xandra

Fast, simple, and robust Cassandra/ScyllaDB driver for Elixir.
ISC License
402 stars 54 forks source link

Fix monitor leak in Xandra.Connection #351

Closed mattias01 closed 1 year ago

mattias01 commented 1 year ago

When calling prepare or execute on a connection, a monitor is started on that connection which is configured to be removed when it receives a reply using the created alias. However it does not always receive a reply using that alias, which can lead to a resource leak where monitors are created and never removed.

This is mostly (only?) a problem when calling prepare on a long running connection from a long running process when the prepared query already is in the cache, because then the connection does not send a reply using the alias. The leaked monitors eventually takes up enough memory to crash the application. In my case, after a few hours.

Example before fix:

iex> Process.info(pid(0, 704 ,0), :dictionary) |> elem(1) |> Enum.take(1)
["$initial_call": {Xandra.Connection, :init, 1}]
iex> Process.info(pid(0, 704 ,0), :monitored_by) |> elem(1) |> length()
105236

By explicitly demonitoring when a reply is not guaranteed, the resource leak is fixed.

Example after fix:

iex> Process.info(pid(0, 722 ,0), :dictionary) |> elem(1) |> Enum.take(1)
["$initial_call": {Xandra.Connection, :init, 1}]
iex> Process.info(pid(0, 722 ,0), :monitored_by) |> elem(1) |> length()
0
whatyouhide commented 1 year ago

This is a great find, thank you @mattias01 🙏 Will release 0.18.1 soon.