p / redis-dump-load

Dump redis databases, load data into redis - in Python
BSD 2-Clause "Simplified" License
175 stars 76 forks source link

redis-load is not adding the data to redis due to invalid ttl value #62

Closed shridharcs closed 3 years ago

shridharcs commented 5 years ago

Hello,

I am new to use this utility. I am seeing an issue in which dump works well but load is not working or in other words redis set is not working.

My dump file looks as follows. ./redisdl.py --pretty > dump.json { "myKey": { "expireat": 1565594482.6172411, "ttl": -0.001, "type": "string", "value": "10" }, "myhash": { "expireat": 1565594482.61765, "ttl": -0.001, "type": "hash", "value": { "value": "key" } } }

So while adding since ttl is -1 it is not able to set the data in redis. ./redisdl.py -l dump.json

This is due to this. https://github.com/p/redis-dump-load/blob/master/redisdl.py#L490 r.pexpire_or_expire_pipeline(p, key, ttl)

If i comment the line then the data insertion is OK. Basically what the function pexpire_or_expire_pipeline does is return p.pexpire(key, int(ttl * 1000)), so ttl is set to -1 and hence the key is expired.

So could someone tell me how to get around this issue? Or do i need to pass something else during dump, i didnt see any in help text/code.

Same issue is seen even with --use-expireat as well.

Is this known issue or am i missing something?

shridharcs commented 5 years ago

hello,

Issue is due to this. Simple working example,

import redis r = redis.StrictRedis(host="localhost", port=6379, db=0) p = r.pipeline(transaction=False) p.set("Shridhar", "CS") Pipeline<ConnectionPool<Connection>> p.execute() [True] r.get("Shridhar") 'CS'

Non Working,

import redis r = redis.StrictRedis(host="localhost", port=6379, db=0) p = r.pipeline(transaction=False) p.set("Shridhar", "CS") Pipeline<ConnectionPool<Connection>> p.pexpire("Shridhar", int(-0.001 * 1000)) Pipeline<ConnectionPool<Connection>> p.execute() [True, 1L] r.get("Shridhar")

This is evident from this as well. 127.0.0.1:6379> set Shridhar CS OK 127.0.0.1:6379> get Shridhar "CS" 127.0.0.1:6379> PEXPIRE Shridhar -1 (integer) 1 127.0.0.1:6379> get Shridhar (nil) 127.0.0.1:6379>

So the issue is with pexpire. Can someone tell me whether we need to update this code?

shridharcs commented 5 years ago

Could some expert comment on this?

p commented 5 years ago

It sounds like this is the same issue as https://github.com/p/redis-dump-load/issues/60.

hnimminh commented 3 years ago

just add an pull request to fix this issue https://github.com/p/redis-dump-load/pull/64. I make some test, that fix work well for me. @shridharcs can confirm if this fix work for you.

shridharcs commented 3 years ago

@hnimminh this works.