search

trinodb / trino-python-client

Python client for Trino
Apache License 2.0
311 stars 154 forks source link

Add a configurable sleep to query status polling #193

Closed nickwareing closed 2 years ago

nickwareing commented 2 years ago

Currently, the client loops continuously whilst a query is running, sending GET fetch requests. For clients issuing a high volume of long running queries (ETL), these requests add unnecessary load to the Trino cluster and any intermediary proxying systems (lyft-gateway for example).

Can we add a configurable sleep (default to 0 to preserve current behaviour) to the fetch logic?

dain commented 2 years ago

This is how the Trino protocol is designed to work. The server assumes that clients are dumb and delays responses to force request waits.

nickwareing commented 2 years ago

Closing, based on @dain's explanation.