Open narayanacharya6 opened 2 years ago
Managed to isolate the instance where the above process is getting stuck. It blocks after it reads this from the buffer:
b'+\'South Korea s SK Innovation to file patent infringement lawsuit against LG Chem in U.S. South Korea\'"\'"\'s SK Innovation to file patent infringement lawsuit against LG Chem in U.S. \n'
This is the value printed in the data
variable at line https://github.com/redis/redis-py/blob/master/redis/connection.py#L249
Removing the field from my query no longer blocks socket IO. Any ideas why this might be happening?
Also, let me know if I should rather move this issue to the redis-py project.
Hi @narayanacharya6 thank you for reporting! would you mind sharing a dataset with which we can reproduce this behavior?
Hey @swilly22, due to data privacy reasons I am unable to share the data/query that led to this. I will try to reproduce this issue with dummy data and report back. Meanwhile, do you mind answering a couple of questions, which I anticipate to be the culprit (but have no proof of at the moment)?
Can having \n
(like above) or \r
or other special characters in my response fields lead to inconsistent behavior with reading from the connection buffer? I see that these characters are used as separators while reading fields (https://github.com/redis/redis-py/blob/master/redis/connection.py#L71) .
When connecting over unix sockets, are there limits to how long the requested query can be (mine is upwards of 20k string characters, UNION
of many MATCH
queries) or how large the response can be? If there are buffer limits and associated configs, can you point me to some documentation?
FYI, these queries work as expected when connecting via TCP.
Ok, I think its because of \n
in the response fields.
Minimal example to reproduce inspired by the getting started example here - https://docs.redis.com/latest/modules/redisgraph/redisgraph-quickstart/ is below. The only change I've introduced in the example is added \n
to the name Valentino Rossi
.
Start your redis-server with redisgraph module and unix socket.
redis-server --unixsocket /tmp/redis.sock --loadmodule /path/to/redisgraph.so
For redis-cli:
a. Start redis-cli: redis-cli -s /tmp/redis.sock
b. Create graph: GRAPH.QUERY MotoGP "CREATE (:Rider {name:'Valentino Rossi\n'})-[:rides]->(:Team {name:'Yamaha'}), (:Rider {name:'Dani Pedrosa'})-[:rides]->(:Team {name:'Honda'}), (:Rider {name:'Andrea Dovizioso'})-[:rides]->(:Team {name:'Ducati'})"
c. Run query: GRAPH.QUERY MotoGP "MATCH (r:Rider)-[:rides]->(t:Team) WHERE t.name = 'Yamaha' RETURN r,t"
You will get a response:
1) 1) r
2) t
2) 1) 1) 1) 1) "id"
2) (integer) 0
2) 1) "labels"
2) 1) Rider
3) 1) "properties"
2) 1) 1) name
2) Valentino Rossi
2) 1) 1) "id"
2) (integer) 1
2) 1) "labels"
2) 1) Team
3) 1) "properties"
2) 1) 1) name
2) Yamaha
3) 1) "Cached execution: 0"
2) "Query internal execution time: 0.526000 milliseconds"
import redis
from redisgraph import Graph
redis_conn = redis.Redis(host="localhost", port=6379, decode_responses=True, unix_socket_path="/tmp/redis.sock")
graph = Graph("MotoGP", redis_conn)
graph.query("MATCH (r:Rider)-[:rides]->(t:Team) WHERE t.name = 'Yamaha' RETURN r,t")
The above never returns.
This is what I'm getting on my dev machine:
BigSur 11.6.1
Python 3.9.5
RedisGraph master branch
pip list | grep redis
hiredis 2.0.0
redis 3.5.3
redisgraph 2.4.1
Test script:
import redis
from redisgraph import Graph
db_name = "MotoGP"
def query_db(connection):
graph = Graph(db_name, connection)
# query graph
q = """MATCH (r:Rider)-[:rides]->(t:Team) WHERE t.name = 'Yamaha' RETURN r,t"""
result = graph.query(q)
result.pretty_print()
#-------------------------------------------------------------------------------
# form connections
#-------------------------------------------------------------------------------
# connect to database using TCP
tcp_conn = redis.Redis(host='localhost', port=6379, decode_responses=True)
# connect to database using unix-socket
unix_socket_conn = redis.Redis(host='localhost', port=6379, decode_responses=True, unix_socket_path="/tmp/redis.sock")
#-------------------------------------------------------------------------------
# populate graph
#-------------------------------------------------------------------------------
tcp_conn.flushall()
q = """CREATE (:Rider {name:'Valentino Rossi\n'})-[:rides]->(:Team {name:'Yamaha'})"""
#graph = Graph(db_name, tcp_conn)
graph = Graph(db_name, unix_socket_conn)
result = graph.query(q)
#-------------------------------------------------------------------------------
# query graph
#-------------------------------------------------------------------------------
query_db(tcp_conn)
query_db(unix_socket_conn)
Results:
python3 test.py
+------------------------+-----------------+
| r | t |
+------------------------+-----------------+
| {name:"Valentino Rossi | {name:"Yamaha"} |
| "} | |
+------------------------+-----------------+
Cached execution 0.0
internal execution time 0.358
+------------------------+-----------------+
| r | t |
+------------------------+-----------------+
| {name:"Valentino Rossi | {name:"Yamaha"} |
| "} | |
+------------------------+-----------------+
Cached execution 1.0
internal execution time 0.149
Both connections produce the same result-set, I'm adding @chayim to this issue maybe he'll have an idea,
@narayanacharya6 would you mind installing hiredis
when this python package is available redis-py
switches to it in-order to perform redis response parsing.
@swilly22 Installing hiredis
makes the difference. With hiredis, I do get back responses as expected.
Keeping this issue open in case we expect the same behavior without hiredis
as well.
Thank you @narayanacharya6 for validating this, we will look into the none hiredis
case
Will you be running with hiredis
?
@chayim do you think \n
messes with the python response parsing ?
@swilly22 For now, I can continue working with hiredis
installed. I haven't done rigorous testing yet - but so far seems fine.
Response time out with client connected using
unix_socket_path
when using python client but not withredis-cli
.I have a
redis-server
instance running locally accepting connections via unix socket.redisgraph.so
was built by compiling the RedisGraph project.redis-server
was started as:I create the python client as:
My queries to construct the nodes and edges of the graph work fine. But when I try to to query for something from the constructed graph, the response never returns (partial stack trace below). When I try to run the same query from
redis-cli
(run asredis-cli -s /tmp/redis.sock
) I do get a response. Leads me to believe something in the python client is not right, maybe.Other Details: OS version: macOS Monterey 12.0.1, Intel-based MBP (2019)
pip version of clients used:
redis-server version:
Redis server v=6.2.6 sha=00000000:0 malloc=libc bits=64 build=c6f3693d1aced7d9
redisgraph.so was built from latest master branch.