TypeError in parse_m_get

[ilieszidane]

Hi, I am not sure if I am doing something wrong here but it seems like this is a bug in parse_m_get.

I am created a single time series with few tags and added some samples. Now I want to fetch the latest sample per tags using mget.

redis.create('my_series', labels={'ext_id': 'abcd', 'source': 'my_favorite_src'})
redis.add('my_series', 0, 1)
redis.add('my_series', 1, 2)
data = redis.mget(['ext_id=abcd'])  # <-- this raises TypeError

In line 69 of client.py there is:

res.append({ nativestr(item[0]) : [list_to_dict(item[1]),
                                int(item[2][0]), float(item[2][1])]})

The int(item[2][0]) is raising the following exception. Here item[2] contains a timestamp.

Exception has occurred: TypeError
'int' object is not subscriptable
  File "/usr/local/pip-global/redistimeseries/client.py", line 69, in parse_m_get
    int(item[2][0]), float(item[2][1])]})
  File "/usr/local/pip-global/redis/client.py", line 921, in parse_response
    return self.response_callbacks[command_name](response, **options)
  File "/usr/local/pip-global/redis/client.py", line 901, in execute_command
    return self.parse_response(conn, command_name, **options)
  File "/usr/local/pip-global/redistimeseries/client.py", line 472, in mget
    return self.redis.execute_command(self.MGET_CMD, *params)
  File "/workspaces/quantapp-volmonitor/volmonitor/command/download.py", line 19, in <module>
    data = redis.mget(['ext_id=abcd'])

response = [['my_series', [['ext_id', 'abcd'], ['source', 'my_favorite_src']], 1, '2']]

Please let me know if more details are required.

Thanks for your help.

[filipecosta90]

Hi there @ilieszidane I've tried your example and it all seems to work on my side: Sample code:

from redistimeseries.client import Client

rts = Client()
rts.create('my_series', labels={'ext_id': 'abcd', 'source': 'my_favorite_src'})
rts.add('my_series', 0, 1)
rts.add('my_series', 1, 2)
data = rts.mget(['ext_id=abcd'])
print(data)

## bonus -- check redis and redistimeseries version
from redis import Redis
native_client = Redis()
modules_info = native_client.info("modules")
print( "Modules present in redis: {}".format(modules_info) )

$ python3 rts_82.py 
[{'my_series': [{}, 1, 2.0]}]
Modules present in redis: {'module': {'name': 'timeseries', 'ver': 10406, 'api': 1, 'filters': 0, 'usedby': '[]', 'using': '[]', 'options': '[]'}}

---

Further info required

Can you please provide the RedisTimeSeries version you're using and the redistimeseries-py version as well? To quickly get the package version do as follows:

$ pip list | grep redistimeseries
redistimeseries        1.4.3

[ilieszidane]

Hi Filipe, Apologies, I should have given this before.

I am running redislabs/redisedge:latest as Redis server.

>> module list

1) 1) "name"
   2) "ai"
   3) "ver"
   4) (integer) 400
2) 1) "name"
   2) "rg"
   3) "ver"
   4) (integer) 999999
3) 1) "name"
   2) "timeseries"
   3) "ver"
   4) (integer) 999999

but from the client:

Modules present in redis: {}

Now I am confused.

pip3 freeze | grep redis
hiredis==1.1.0
redis==3.5.3
redistimeseries==1.4.3

Thanks for your help!

[filipecosta90]

Hi there @ilieszidane , it seems that your redisedge container is a bit outdated in the versions it contains. Is it possible to pull the latest one via:

docker pull redislabs/redisedge:latest

You should then have the following modules available:

$ redis-cli
127.0.0.1:6379> module list
1) 1) "name"
   2) "ai"
   3) "ver"
   4) (integer) 10002
2) 1) "name"
   2) "timeseries"
   3) "ver"
   4) (integer) 10406
3) 1) "name"
   2) "bf"
   3) "ver"
   4) (integer) 20204
4) 1) "name"
   2) "rg"
   3) "ver"
   4) (integer) 10003
5) 1) "name"
   2) "search"
   3) "ver"
   4) (integer) 20003
6) 1) "name"
   2) "graph"
   3) "ver"
   4) (integer) 20208
7) 1) "name"
   2) "ReJSON"
   3) "ver"
   4) (integer) 10006

If not then I can try to find the exact image you're using and check why its failing :)

[ilieszidane]

Hi @filipecosta90, Unfortunately I am running the latest: Digest: sha256:f6fd627e711cf467a95d7bcbc2bc7cddb98213897e445f2443a5cf5774dfe6af.

PS C:\Users\izidane> docker pull redislabs/redisedge:latest
latest: Pulling from redislabs/redisedge
f5d23c7fed46: Pull complete                                                                                             5d1536028276: Pull complete                                                                                             84e504b50c70: Pull complete                                                                                             5e92c5cf44cf: Pull complete                                                                                             60295726bf09: Pull complete                                                                                             1f71e00d6f3e: Pull complete                                                                                             6511538c1dbc: Pull complete                                                                                             57c694ea7d43: Pull complete                                                                                             0f6a4b505032: Pull complete                                                                                             fe352c19f3c1: Pull complete                                                                                             59764a043911: Pull complete                                                                                             2a65d86d8251: Pull complete                                                                                             Digest: sha256:f6fd627e711cf467a95d7bcbc2bc7cddb98213897e445f2443a5cf5774dfe6af
Status: Downloaded newer image for redislabs/redisedge:latest
docker.io/redislabs/redisedge:latest
PS C:\Users\izidane> docker run -it --rm -p 6379:6379  redislabs/redisedge:latest
1:C 30 Nov 2020 10:25:39.140 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 30 Nov 2020 10:25:39.140 # Redis version=5.0.5, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 30 Nov 2020 10:25:39.140 # Configuration loaded
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 5.0.5 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 1
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           http://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

1:M 30 Nov 2020 10:25:39.142 # Server initialized
1:M 30 Nov 2020 10:25:39.142 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:M 30 Nov 2020 10:25:39.142 * Module 'ai' loaded from /usr/lib/redis/modules/redisai.so
1:M 30 Nov 2020 10:25:39.142 * Module 'timeseries' loaded from /usr/lib/redis/modules/redistimeseries.so
1:M 30 Nov 2020 10:25:39.144 * <rg> RedisGears version 99.99.99, git_sha=
1:M 30 Nov 2020 10:25:39.144 * <rg> PythonHomeDir:/opt/redislabs/lib/modules/python3
1:M 30 Nov 2020 10:25:39.144 * <rg> MaxExecutions:1000
1:M 30 Nov 2020 10:25:39.144 * <rg> ProfileExecutions:0
1:M 30 Nov 2020 10:25:39.144 * <rg> PythonAttemptTraceback:1
1:M 30 Nov 2020 10:25:39.144 * <rg> RedisAI api loaded successfully.
could not initialize RediSearch_CheckApiVersionCompatibility
1:M 30 Nov 2020 10:25:39.144 # <rg> could not initialize RediSearch api, running without Search support.
1:M 30 Nov 2020 10:25:39.163 * <rg> Initializing Python environment with: exec(open('/opt/redislabs/lib/modules/python3/.venv/bin/activate_this.py').read(), {'__file__': '/opt/redislabs/lib/modules/python3/.venv/bin/activate_this.py'})
1:M 30 Nov 2020 10:25:39.228 * Module 'rg' loaded from /usr/lib/redis/modules/redisgears.so
1:M 30 Nov 2020 10:25:39.228 * Ready to accept connections

module list returns the same as above.

[filipecosta90]

Hi again @ilieszidane you're totally right. I was using the redislabs/redismod:latest and not redislabs/redisedge:latest. Therefore, we have an issue on the redisedge:latest image that we will solve asap. Sorry for the confusion on my side.

In the meaning can you use redislabs/redismod:latest for your experiments? It should have all modules that you wanted by running redisedge plus the all the others officially supported by redislabs.

I'll keep this issue open while we solve the redislabs/redisedge:latest issue.

[ilieszidane]

Thanks very much @filipecosta90 for your help. Using redislabs/redismod:latest indeed solves the problem.

[filipecosta90]

Still valid: edge:latest output:

127.0.0.1:6379> "TS.MGET" "FILTER" "ext_id=abcd"
1) 1) "my_series"
   2) 1) 1) "ext_id"
         2) "abcd"
      2) 1) "source"
         2) "my_favorite_src"
   3) (integer) 1
   4) "2"

redismod:edge ( same as rts ) :

127.0.0.1:6379> "TS.MGET" "FILTER" "ext_id=abcd"
1) 1) "my_series"
   2) (empty array)
   3) 1) (integer) 1
      2) "2"