Open mcrumiller opened 6 hours ago
Does this still happen with the nightly Python builds? Since the last release we did a heavy refactor of some of those internals to make those types of errors report better/go away:
pip install \
--pre \
--index-url https://repo.fury.io/arrow-adbc-nightlies \
adbc-driver-postgresql
Sorry, I'm unable to build the nightlies at work. Unsure if you want me to re-open after the next release if I still see the issue, or if you want to just leave this one in limbo until then.
Or actually, is there a pip release for the nightlies?
I think there's wheels! For me it resolves to https://repo.fury.io/arrow-adbc-nightlies/-/ver_1QpKJK/adbc_driver_postgresql-1.3.0-py3-none-macosx_11_0_arm64.whl
but no idea if that will work for you. Installing dev versions is not great right now, sorry!
If you're up for another approach, figuring out which row/column causes the issue by manipulating the LIMIT clause might help pinpoint the issue. The error that was obscured by the last report of this was one where a computation failed on a specific value.
What happened?
This is a semi-duplicate of #2133. That issue was closed by a PR that apparently improves the error message, but it seems as though it doesn't resolve the issue itself in that instance.
In my case, I'd like to diagnose what's causing the error (if possible) as I'd like to actually get this working--if there's a workaround possible on my end I would like to pursue that possibility before waiting for a new version of ADBC. I'm not sure where the ADBC log file are stored and I can't find in the documentation where I might look.
When I run my query (via polars), I get:
If I add a
LIMIT 10000
the query succeeds, so the issue is either in a later record in the data, or something else that I can't think of. I expect 1,219,228 total records.. Can someone possibly help me diagnose the issue?Stack Trace
How can we reproduce the bug?
It's a fairly complex query. I could perhaps work to reproduce but since it works on 10k records, it may be difficult to make a repro.
Environment/Setup