search

trinodb / trino-python-client

Python client for Trino
Apache License 2.0
311 stars 154 forks source link

Fix issue of queries not reaching completed state #210

Closed guyco33 closed 1 year ago

guyco33 commented 1 year ago

Fix issue of queries not reaching completed state - see discussion on https://trinodb.slack.com/archives/CGB0QHWSW/p1658934510749439

Adds a test to demonstrate effect of https://github.com/trinodb/trino/pull/13055

hashhar commented 1 year ago

@mdesmet @ebyhr Can you PTAL again. This just adds a regression test now (which asserts the broken behaviour) - the fix will be part of #220.

mdesmet commented 1 year ago

This code can be used to simulate this using dbapi on the #220 branch. This didn't result in any unfinished queries. Now again I don't know enough about the related Trino internals what's the chance of this becoming a flaky test on github CI.

import trino

count_not_finished = 0
trino_connection = trino.dbapi.connect(
    host="localhost",
    port=8080,
    user="admin"
)
for _ in range(0, 10000):
    cur = trino_connection.cursor()
    cur.execute("SELECT VERSION()")
    cur.fetchone()
    cur.fetchone() # simulate sqlalchemy scalar_one
    if not cur._query.finished:
        count_not_finished += 1

print(str(count_not_finished) + " unfinished queries")

In other words we need to be sure that doing 2 fetchone() calls (actually fetching up to 3 requests as long next_uri is not null) is enough to fully consume all results and mark the query as finished.

hashhar commented 1 year ago

This has been addressed via #220 now. It has a different test though since the test here is potentially flaky (and also fails some of the existing tests regarding INSERTs).