TTL expiry query

[churcho]

I think I got this wrong. My understanding was this: ttl_check: :timer.seconds(15),

To me, that means the duration of the interval it takes to run/rerun the cache cleanup for expired cache

So in my code, I am setting the expiry to about 30days, I wanted this code to run every say 15 seconds and ONLY delete the expired cache. It is not working that way, what it's doing is deleting all my caches irrespective of their expiry time.

What was the intended use case for this and how can I achieve my original goal?

[sasa1977]

To me, that means the duration of the interval it takes to run/rerun the cache cleanup for expired cache

That's how it's supposed to work. So it checks every 15 secs, and expires items after about 30 days. It's described in more details here.

It is not working that way, what it's doing is deleting all my caches irrespective of their expiry time.

This definitely doesn't sound right. Could you set up a simple example which demonstrates this, so I can take a closer look?

[churcho]

This is what I have in my application file:

 worker(ConCache, [[callback: fn(operation) ->
                          MyApp.Token.save_jwt(operation) end,
                         ttl_check: :timer.seconds(5),
                        ], [name: :jwt_cache]]),

I am saving my cache using this (ttl is set to maybe 120): ConCache.put(:my_cache, key, %ConCache.Item{value: jwt, ttl: ttl})

When I run ConCache.get(:my_cache, key) I get back nil

I have matching blocks for save_jwt handling both :update and :delete and it shows that these are getting called.

[sasa1977]

ttl values are in milliseconds. If you set ttl to 120, it means 120 milliseconds, so the item is going to expire in the next check. I suggest using :timer functions for all the values.

Here's an example that works on my end:

defmodule Test.Application do
  use Application

  def start(_type, _args) do
    import Supervisor.Spec, warn: false
    children = [
      worker(ConCache, [
        [
          callback: fn(operation) -> IO.puts "#{NaiveDateTime.utc_now()} #{inspect(operation)}" end,
          ttl_check: :timer.seconds(1),
        ],
        [name: :jwt_cache]]
      )
    ]

    opts = [strategy: :one_for_one, name: Test.Supervisor]
    res = Supervisor.start_link(children, opts)
    ConCache.put(:jwt_cache, :foo, %ConCache.Item{value: :bar, ttl: :timer.seconds(5)})
    res
  end
end

# 2017-03-24 12:11:38.060336 {:update, #PID<0.168.0>, :foo, :bar}
# 2017-03-24 12:11:44.059523 {:delete, #PID<0.168.0>, :foo}

[sasa1977]

So in my code, I am setting the expiry to about 30days

Also, this is going to be pretty tricky. It's going to work properly only if you don't restart your system for 30 days. Otherwise, when you restart, you can reprime your cache from the database and set TTL to 30 days, but that's going to prolongate their lifetime.

If you don't populate your cache from the database on restart, then you should be fine.

If you do want your items to be deleted from the database after 30 days, then I suggest tackling the problem at the database level, not cache. So basically have some periodic job (say powered by quantum) that checks database items, say once per day, and deletes expired ones.

[churcho]

Also, this is going to be pretty tricky. It's going to work properly only if you don't restart your system for 30 days. Otherwise, when you restart, you can reprime your cache from the database and set TTL to 30 days, but that's going to prolongate their lifetime.

That is where my earlier question on get_or_store comes from :-)

[sasa1977]

That is where my earlier question on get_or_store comes from :-)

:-) Right, so as discussed in that other issue, I think that in your case it's best to sync cache with the database, not the other way around. So you're doing updates/deletes on the database, and then bring the cache up to speed.