centrifugal / centrifugo

Scalable real-time messaging server in a language-agnostic way. Self-hosted alternative to Pubnub, Pusher, Ably. Set up once and forever.
https://centrifugal.dev
Apache License 2.0
8.3k stars 587 forks source link

Message history is not retained after exiting centrifugo and coming back later #113

Closed lmj0011 closed 7 years ago

lmj0011 commented 7 years ago

I am using the redis engine The machine I have my app running in, stays powered on.

The problem is, I have a need to retrieve history messages in my chat app, whenever a user logs in. I use the history method in the centrifuge.js lib to do so. However, history does not seem to be retained in redis for the time indicated in the config file below.

note: 604800 seconds = 1 week

this is how my configuration is set:

{
  "secret": "xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx",
  "connection_lifetime": 7200,
    "publish": true,
    "watch": true,
    "presence": true,
    "join_leave": true,
    "history_size": 500,
    "history_lifetime": 604800,

    "namespaces": [
        {
          "name": "news",
          "publish": true,
          "presence": true,
          "join_leave": true,
          "history_size": 500,
          "history_lifetime":604800,
          "recover": true
        }
    ]
}

Message history disappears from redis in a short amount of time (20 mins or so) after leaving my app for some time and then coming back. note: I'm the only active user in this issue

FZambia commented 7 years ago

Hi, this is strange. Could you try to use ttl Redis command to see what happens with time?

ttl centrifugo.history.list.centrifugo.message.YOUR_CHANNEL_NAME_HERE

It shows how long Redis will keep that key

FZambia commented 7 years ago

Another suggestion - maybe you have memory limit in Redis and some eviction strategy? See http://redis.io/topics/lru-cache - so Redis starts to remove keys to find a space for new data?

lmj0011 commented 7 years ago

Hi, this is strange. Could you try to use ttl Redis command to see what happens with time?

127.0.0.1:6379> ttl centrifugo.history.list.centrifugo.message.$dev+user3
(integer) 598962

maybe you have memory limit in Redis and some eviction strategy?

I have nothing set in my redis.conf

FZambia commented 7 years ago

You can also use llen to see that there are still items in history list:

llen centrifugo.history.list.centrifugo.message.$dev+user3

Don't know why this could happen at moment - Centrifugo never deletes history keys, just sets expire for them. I tried to push some messages into channel before going to sleep - now in the morning history is correct with correct expire ttl.

FZambia commented 7 years ago

Maybe sth happens with Redis itself? After loosing history try to run Redis INFO command in redis-cli. It can show some useful info - uptime_in_seconds, used_memory, evicted_keys etc

lmj0011 commented 7 years ago

Maybe sth happens with Redis itself? After loosing history try to run Redis INFO command in redis-cli. It can show some useful info - uptime_in_seconds, used_memory, evicted_keys etc

I recently changed my history_lifetime to 604800 and restarted centrifugo. Message history looks to be retained, even after I restarted my machine.

So this leads my to think there is a max amount of seconds that centrifugo will take for the history_lifetime. This should be documented if so.

lmj0011 commented 7 years ago

I going to try to use this number 7889231, 3 months in seconds, for history_lifetime. That number shouldn't be a problem, right?

This may not even be an issue, because in my config file I discovered the history_lifetime was set for 1800, 30 minutes, which would explain why it was disappearing so quickly.

FZambia commented 7 years ago

Ah, I am confused a bit - so you found that history_lifetime was set to 1800 seconds which explains why it was cleaned so quickly, right?

Centrifugo does not limit this value (maybe only by maximum possible integer value which is much bigger than you want to set).

lmj0011 commented 7 years ago

Yes that's correct. I think it's safe to close this issue

FZambia commented 7 years ago

Ok, cool!:)