Attumm / redis-dict

Python dictionary with Redis as backend, built for large datasets. Simplifies Redis operations for large-scale and distributed systems. Supports various data types, namespacing, pipelining, and expiration.
https://attumm.github.io/redis-dict/
MIT License
53 stars 14 forks source link

v1.5.2 retrieves list as str #10

Closed Girgitt closed 2 years ago

Girgitt commented 3 years ago

Maybe I am doing something wrong but storing simple list is not possible with version 1.5.2 Retrieved value is casted to str.

>>> d[1] = list([1 for i in range(10)])
>>> d
{'1': '[1, 1, 1, 1, 1, 1, 1, 1, 1, 1]'}
>>> d[1]
'[1, 1, 1, 1, 1, 1, 1, 1, 1, 1]'
Girgitt commented 3 years ago

value saved to redis seems to have a proper type preserved: image

Girgitt commented 3 years ago

Description from readme explains this behavior:

redis-dict can be used as a drop-in replacement for a normal dictionary as long as no datastructures are used by reference. i.e. no nested layout e.g. values such list,.

So it's not a bug, it's a feature type of thing..

Attumm commented 3 years ago

Hi @Girgitt,

First of thank you for bring this up, If you don't mind I'll reopen the ticket.

If the need is there to store a list then we should add it as a feature.

For a while there have been some discussions about storing referenced types too. But there are performance penalties for doing it. The initial start of project was for big data ("more data then local ram") kind of applications. speed, and memory have been at goal, Therefor it was better not to store lists, or dicts. But this is not case for all projects. So let's add it.

Attumm commented 3 years ago

New version 1.6.0 with experimental support for list, and dicts.

>>> dic = RedisDict(namespace='bar')
>>> dic['a'] = ['a', 'b', 'c']
>>> dic['a']
['a', 'b', 'c']

Let me know if this solves your use-case.

https://pypi.org/project/redis-dict/1.6.0/

Girgitt commented 2 years ago

@Attumm thanks for a quick response. I already changed my use case to avoid storing lists but will take another shot soon and let you know how it worked.

Attumm commented 2 years ago

@Girgitt Any luck with testing?

I rather not close the issue myself.

Attumm commented 2 years ago

A version has been released that does offer a initial support for lists and dicts.

Girgitt commented 2 years ago

@Attumm thank you for the change. I used the ability to overload serialization method and used your lib to handle lists just fine. So for my use case, the functionality is sufficient - in fact, redis-dict makes the scale-up of FastAPI straightforward and easy.