sfu-db / connector-x

Fastest library to load data from DB to DataFrames in Rust and Python
https://sfu-db.github.io/connector-x
MIT License
1.85k stars 146 forks source link

Any Query hangs forever on Postgresql in AWS RDS Aurora #549

Open sduc opened 8 months ago

sduc commented 8 months ago

What language are you using?

Python

What version are you using?

0.3.2

What database are you using?

PostgreSQL (15.3) on AWS RDS Aurora Standard.

What dataframe are you using?

Arrow (but same happens with pandas)

Can you describe your bug?

I'm doing the following simple query but the same happens for any other queries I tried:

connection_string = f"postgresql://{uriEncodedUser}:{uriEncodedPassword}@{host}:5432/main"
cx.read_sql(connection_string, "SELECT 1", return_type="arrow2")

The execution hangs forever and never completes.

When enabling debug mode with:

import os
os.environ["RUST_LOG"] = "connectorx=debug,connectorx_python=debug"

This is what I'm seeing before it hangs

[2023-11-01T11:05:09Z DEBUG connectorx::get_arrow2] Protocol: binary
[2023-11-01T11:05:11Z DEBUG connectorx::dispatcher] Run dispatcher
[2023-11-01T11:05:11Z DEBUG connectorx::dispatcher] Prepare
[2023-11-01T11:05:11Z DEBUG connectorx::dispatcher] Fetching metadata
[2023-11-01T11:05:11Z DEBUG connectorx::dispatcher] Do not need counts in advance
[2023-11-01T11:05:11Z DEBUG connectorx::dispatcher] Allocate destination memory: 0x1
[2023-11-01T11:05:11Z DEBUG connectorx::dispatcher] Create destination partition
[2023-11-01T11:05:11Z DEBUG connectorx::dispatcher] Start writing
[2023-11-01T11:05:11Z DEBUG connectorx::dispatcher] Finalize partition 0
[2023-11-01T11:05:11Z DEBUG connectorx::dispatcher] Partition 0 finished

What are the steps to reproduce the behavior?

Database setup if the error only happens on specific data or data type

The dabase in question is AWS RDS Aurora Standard Postgresql.

Note: I've tried with a regular postgres on my local machine and I do not have this issue.

Example query / code

any query will do the trick.

What is the error?

No error. The code just hangs.