Closed jakob-keller closed 1 month ago
@chayim This is the bug we discussed.
here's the bt:
Program received signal SIGSEGV, Segmentation fault.
0x0000aaaaaab9780c in PyTuple_GetItem ()
(gdb) bt
#0 0x0000aaaaaab9780c in PyTuple_GetItem ()
#1 0x0000fffff6b05920 in tryParentize (obj=0xfffff6735c40,
task=0xaaaaab36b5d0) at src/reader.c:81
#2 createArrayObject (task=0xaaaaab36b5d0, elements=<optimized out>)
at src/reader.c:171
#3 0x0000fffff6b0cb18 in processAggregateItem (r=0xaaaaab334e60)
at vendor/hiredis/read.c:540
#4 processItem (r=0xaaaaab334e60) at vendor/hiredis/read.c:647
#5 redisReaderGetReply (r=0xaaaaab334e60, reply=reply@entry=0xffffffffd0a8)
at vendor/hiredis/read.c:763
#6 0x0000fffff6b04acc in Reader_gets (self=0xfffff671b7d0,
args=<optimized out>) at src/reader.c:377
#7 0x0000aaaaaab88d60 in ?? ()
#8 0x0000aaaaaab8d3f8 in _PyEval_EvalFrameDefault ()
#9 0x0000aaaaaabb4148 in ?? ()
#10 0x0000aaaaaab8e2ac in _PyEval_EvalFrameDefault ()
#11 0x0000aaaaaaba5348 in _PyFunction_Vectorcall ()
#12 0x0000aaaaaab8d3f8 in _PyEval_EvalFrameDefault ()
#13 0x0000aaaaaabb4288 in ?? ()
#14 0x0000aaaaaab8ed90 in _PyEval_EvalFrameDefault ()
#15 0x0000aaaaaabb4288 in ?? ()
#16 0x0000aaaaaab8ed90 in _PyEval_EvalFrameDefault ()
#17 0x0000aaaaaabb4288 in ?? ()
--Type <RET> for more, q to quit, c to continue without paging--c
#18 0x0000aaaaaab8ed90 in _PyEval_EvalFrameDefault ()
#19 0x0000aaaaaaba5348 in _PyFunction_Vectorcall ()
#20 0x0000aaaaaab8d3f8 in _PyEval_EvalFrameDefault ()
#21 0x0000aaaaaaba5348 in _PyFunction_Vectorcall ()
#22 0x0000aaaaaab8d3f8 in _PyEval_EvalFrameDefault ()
#23 0x0000aaaaaab99d04 in _PyObject_FastCallDictTstate ()
#24 0x0000aaaaaabb0224 in ?? ()
#25 0x0000aaaaaab9ac18 in _PyObject_MakeTpCall ()
#26 0x0000aaaaaab91af4 in _PyEval_EvalFrameDefault ()
#27 0x0000aaaaaaba5348 in _PyFunction_Vectorcall ()
#28 0x0000aaaaaab99da8 in _PyObject_FastCallDictTstate ()
#29 0x0000aaaaaabb0224 in ?? ()
#30 0x0000aaaaaab9b108 in ?? ()
#31 0x0000aaaaaabb4e0c in PyObject_Call ()
#32 0x0000aaaaaab8ed90 in _PyEval_EvalFrameDefault ()
#33 0x0000aaaaaabb4148 in ?? ()
#34 0x0000aaaaaab9177c in _PyEval_EvalFrameDefault ()
#35 0x0000aaaaaac89760 in ?? ()
#36 0x0000aaaaaac895e4 in PyEval_EvalCode ()
#37 0x0000aaaaaacbccbc in ?? ()
#38 0x0000aaaaaacb53c8 in ?? ()
#39 0x0000aaaaaacbc96c in ?? ()
#40 0x0000aaaaaacbbad4 in _PyRun_SimpleFileObject ()
#41 0x0000aaaaaacbb6a0 in _PyRun_AnyFileObject ()
#42 0x0000aaaaaacabfc0 in Py_RunMain ()
#43 0x0000aaaaaac7a748 in Py_BytesMain ()
#44 0x0000fffff7d173fc in __libc_start_call_main (main=main@entry=0xaaaaaac7a720, argc=argc@entry=2, argv=argv@entry=0xffffffffef58) at ../sysdeps/nptl/libc_start_call_main.h:58
#45 0x0000fffff7d174cc in __libc_start_main_impl (main=0xaaaaaac7a720, argc=2, argv=0xffffffffef58, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=<optimized out>) at ../csu/libc-start.c:392
#46 0x0000aaaaaac7a630 in _start ()
(gdb) py-bt
Traceback (most recent call first):
File "/tmp/y/venv/lib/python3.10/site-packages/redis/_parsers/hiredis.py", line 128, in read_response
response = self._reader.gets()
File "/tmp/y/venv/lib/python3.10/site-packages/redis/connection.py", line 500, in read_response
response = self._parser.read_response(disable_decoding=disable_decoding)
File "/tmp/y/venv/lib/python3.10/site-packages/redis/client.py", line 553, in parse_response
response = connection.read_response()
File "/tmp/y/venv/lib/python3.10/site-packages/redis/cluster.py", line 1151, in _execute_command
response = redis_node.parse_response(connection, command, **kwargs)
File "/tmp/y/venv/lib/python3.10/site-packages/redis/cluster.py", line 1101, in execute_command
res[node.name] = self._execute_command(node, *args, **kwargs)
File "/tmp/y/venv/lib/python3.10/site-packages/redis/commands/core.py", line 792, in command
return self.execute_command("COMMAND", **kwargs)
File "/tmp/y/venv/lib/python3.10/site-packages/redis/_parsers/commands.py", line 70, in initialize
commands = r.command()
File "/tmp/y/venv/lib/python3.10/site-packages/redis/_parsers/commands.py", line 67, in __init__
self.initialize(redis_connection)
File "/tmp/y/venv/lib/python3.10/site-packages/redis/cluster.py", line 645, in __init__
self.commands_parser = CommandsParser(self)
File "/tmp/y/venv/lib/python3.10/site-packages/redis/cluster.py", line 488, in from_url
return cls(url=url, **kwargs)
File "/tmp/y/t.py", line 3, in <module>
client = redis.cluster.RedisCluster.from_url(url)
Hi @jakob-keller ,
This is indeed the same as #3145. From the output I can say for sure it is the hiredis-py
issue with maps inside sets. If you can uninstall hiredis-py
, it will solve the problem. Bug report for the root cause: https://github.com/redis/hiredis-py/issues/188
I'm also curious, what is your main reason for using hiredis-py
instead of the built-in RESP parser from redis-py
?
Thanks for following up!
I'm also curious, what is your main reason for using
hiredis-py
instead of the built-in RESP parser fromredis-py
?
My use case is performance critical and this repo's README states:
For faster performance, install redis with hiredis support, this provides a compiled response parser, and for most cases requires zero code changes.
Thank you for addressing this. A quick test confirms that the issue is fixed by hiredis==3.0.0
.
Version: redis-py 5.0.1 / Redis 7.1 (AWS ElastiCache)
Platform: CPython 3.11.7 on macOS Sonoma 14.2 (M2 Pro)
Description:
redis.cluster.RedisCluster
instances with protocol=3 (i.e. RESP3 enabled) crash the Python interpreter when performing commands and thehiredis
extra dependency is installed in the local environment.Example to reproduce: