sasa1977 / con_cache

ets based key/value cache with row level isolated writes and ttl support
MIT License
910 stars 71 forks source link

Support for persisting across restart/crash #18

Closed mrdziuban closed 7 years ago

mrdziuban commented 8 years ago

Is there any way to persist the ETS table across a restart/crash? Thanks in advance!

sasa1977 commented 8 years ago

ETS tables are memory-only, so it doesn't work out of the box. You could use Mnesia which gives you this for free. Unfortunately, con_cache hardcodes the format of ETS tuples, which are not compatible with Mnesia, so the two won't work together. I'll think about providing some configurability so con_cache can work with Mnesia. Not sure if/when will that happen though.

At the present, you can install a custom callback function which is invoked on every cache modification (see :callback option when starting the cache). In the function, you can then persist the change, for example to DETS or some other external database. That's roughly the approach I took in the system for which I originally designed ConCache. Notice though that this approach won't work if you're bypassing ConCache, i.e. if you're modifying the ets table directly.

To restore the cache when (re)starting the system, I'd suggest a simple wrapper, something like:

defmodule MyPersistableCache
  def start_link(...) do
    case ConCache.start_link(...) do
      {:ok, pid} ->
        restore_cache(pid)
        {:ok, pid}

      error -> error
    end
  end
end

And then you place MyPersistableCache instead of ConCache in the supervision tree.

Finally, you can always wrap each ConCache operation with your own function which modifies the cache and persists the change. This is a bit more cumbersome, because you have to wrap every function. On the upside, you have better control, so you can for example first persist the entry, and only if that succeeds, store it to cache.

Let me know if you have further questions.