Closed vogelsgesang closed 11 months ago
kou
commented
11 months ago Thanks for trying this project!
Did you try it with 0.1.0? If so, could you try the main branch instead? There are many improvements after 0.1.0. (We should release a new version...)
vogelsgesang
commented
11 months ago I tried version 0.1.0, following the instructions from https://arrow.apache.org/flight-sql-postgresql/0.1.0/install.html
vogelsgesang
commented
11 months ago I built from source code now (based on commit 60c748aa1fbf7aa1bee125849bfa9f7635fca5fc). Unfortunately still no success.
Here is the console output which I get
avogelsgesang@avogelsges-wsl2:~/Documents$ python3 af-query.py
/home/avogelsgesang/.local/lib/python3.10/site-packages/adbc_driver_manager/dbapi.py:298: Warning: Cannot disable autocommit; conn will not be DB-API 2.0 compliant
warnings.warn(As soon as I kill Postgres using sudo pkill -9 -u postgres, I get the following stack trace from the client
Traceback (most recent call last):
File "/home/avogelsgesang/Documents/af-query.py", line 9, in <module>
cur.execute("SELECT 42;")
File "/home/avogelsgesang/.local/lib/python3.10/site-packages/adbc_driver_manager/dbapi.py", line 669, in execute
handle, self._rowcount = self._stmt.execute_query()
File "adbc_driver_manager/_lib.pyx", line 1106, in adbc_driver_manager._lib.AdbcStatement.execute_query
File "adbc_driver_manager/_lib.pyx", line 227, in adbc_driver_manager._lib.check_error
adbc_driver_manager.OperationalError: IO: [FlightSQL] error reading from server: EOF (Unavailable; ExecuteQuery)Are there any debug setting which I could enable so I get additional tracing on the server and/or client?
kou
commented
11 months ago log_min_messages=debug5 PostgreSQL configuration helps us but it generates many outputs...
I'll try this later.
FYI: We have a simple ADBC client written in Ruby https://github.com/apache/arrow-flight-sql-postgresql/blob/main/benchmark/select-adbc-flight-sql.rb and it worked when I used. So ADBC client must work.
vogelsgesang
commented
11 months ago Here is the debug output under log_min_messages=debug5
The query from ADBC was sent at 23:04:29 in the log file
vogelsgesang
commented
11 months ago Ok, I think I am getting closer. The problem seems to be that the "parameters" of the prepared query are never set. If I use
cur.execute("SELECT $1;", parameters=(42,))instead of
cur.execute("SELECT 42;")everything works as expected.
For the original, non-parameterized query, I was able to confirm that parameters_ is empty when reaching PreparedStatement::select
I guess ADBC skips the call to DoPutPreparedStatementQuery in this case. Not sure if this is a client-side bug in ADBC. But even if it is, I guess the server should be robust against this and report an error?
kou
commented
11 months ago Ah, good catch! I didn't care about the case. I've fixed it: GH-179
I've opened #180 for a timeout case.
vogelsgesang
commented
11 months ago Thanks for the patch! With this, the line
cur.execute("SELECT 42;")now works as expected.
However, error handling seems to be brittle. The query
cur.execute("SELECT 1/0;")first hangs for a couple of minutes before then reporting
Traceback (most recent call last):
File "/home/avogelsgesang/Documents/af-query.py", line 9, in <module>
cur.execute("SELECT 1/0;")
File "/home/avogelsgesang/.local/lib/python3.10/site-packages/adbc_driver_manager/dbapi.py", line 669, in execute
handle, self._rowcount = self._stmt.execute_query()
File "adbc_driver_manager/_lib.pyx", line 1106, in adbc_driver_manager._lib.AdbcStatement.execute_query
File "adbc_driver_manager/_lib.pyx", line 227, in adbc_driver_manager._lib.check_error
adbc_driver_manager.ProgrammingError: INVALID_ARGUMENT: [FlightSQL] failed to run: Action::SelectPreparedStatement: division by zero (InvalidArgument; ExecuteQuery)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/avogelsgesang/Documents/af-query.py", line 8, in <module>
with conn.cursor() as cur:
File "/home/avogelsgesang/.local/lib/python3.10/site-packages/adbc_driver_manager/dbapi.py", line 225, in __exit__
self.close()
File "/home/avogelsgesang/.local/lib/python3.10/site-packages/adbc_driver_manager/dbapi.py", line 611, in close
self._stmt.close()
File "adbc_driver_manager/_lib.pyx", line 1078, in adbc_driver_manager._lib.AdbcStatement.close
File "adbc_driver_manager/_lib.pyx", line 1084, in adbc_driver_manager._lib.AdbcStatement.close
File "adbc_driver_manager/_lib.pyx", line 227, in adbc_driver_manager._lib.check_error
adbc_driver_manager.ProgrammingError: INVALID_ARGUMENT: [FlightSQL] stolen session: 1283150246607583965 (InvalidArgument; ClosePreparedStatement)
Exception ignored in: <function Cursor.__del__ at 0x7fa517960280>
Traceback (most recent call last):
File "/home/avogelsgesang/.local/lib/python3.10/site-packages/adbc_driver_manager/dbapi.py", line 766, in __del__
File "/home/avogelsgesang/.local/lib/python3.10/site-packages/adbc_driver_manager/dbapi.py", line 611, in close
File "adbc_driver_manager/_lib.pyx", line 1077, in adbc_driver_manager._lib.AdbcStatement.close
File "adbc_driver_manager/_lib.pyx", line 293, in adbc_driver_manager._lib._AdbcHandle._close_child
File "adbc_driver_manager/_lib.pyx", line 295, in adbc_driver_manager._lib._AdbcHandle._close_child
RuntimeError: Underflow in closing this AdbcStatementThanks for finding one more edge case. Could you open a new issue for it to close this by #179?
I tried using this extension through ADBC from Python, using the following script
Observed behavior: The script is hanging forever