Add option to reset cache values for warming

[quolpr]

For now, I am making such thing do remove old cache:

keys =
  case Cachex.keys(:active_publisher_account_cache) do
    {:ok, keys} -> keys
    _ -> []
  end

new_cache = ...

(keys -- (for {key, _} <- new_cache, do: key))
|> Enum.each(&Cachex.del(:active_publisher_account_cache, &1))

{:ok, new_cache}

And I don't really like such approach. I know, that put_many/3 has ttl support, but is still not right thing - race condition can happen(when cache data is expired, but warmer still fetching data. So, there are no data in cache).

I think that best way is to add the new option to put_many/3, like reset: true. It will: 1) Set new cache 2) Delete keys, that are not present in pairs of put_many argument

[whitfin]

@quolpr I'm not sure I follow.

If you have a warmer that runs every 30s you should use a TTL of something like 35s. This will prune any that were not written again, but not those which were added in the last batch.

I am against adding reset: true because it's not really needed; you can accomplish the same using the TTL options.

[quolpr]

If you have a warmer that runs every 30s you should use a TTL of something like 35s. This will prune any that were not written again, but not those which were added in the last batch.

What will happen if warmer will be running 40s(suddenly!)? As I understand in interval 35s..40s there will be no data in cache

[whitfin]

@quolpr so you change the TTL based on the worst case performance of your warmer.

If you absolutely cannot make TTL work for you, then what you're doing is the best option, although I would structure it more like this:

defmodule MyProject.Warmer do
  use Cachex.Warmer

  def interval,
    do: :timer.seconds(30)

  def execute(state) do
    new_entries = do_something()

    new_keys =
        new_entries
        |> Enum.map(&elem(&1, 0))
        |> MapSet.new

    :active_publisher_account_cache
    # stream all cache keys that currently exist  
    |> Cachex.stream(Cachex.Query.unexpired(:key))
    # filter out all keys that will be overwritten
    |> Stream.filter(&!MapSet.member?(new_keys, &1))
    # delete each key that's left (removing the older keys)
    |> Enum.each(&Cachex.del(:active_publisher_account_cache, &1))

    new_entries
  end
end

[whitfin]

I am going to close this as regardless of if we need to do anything to help this use case, adding a reset option to put_many/3 is not the correct course of action.

Please feel free to leave any questions and/or comments.