pydiverse / pydiverse.pipedag

A data pipeline orchestration library for rapid iterative development with automatic cache invalidation allowing users to focus writing their tasks in pandas, polars, sqlalchemy, ibis, and alike.
https://pydiversepipedag.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
15 stars 2 forks source link

Fix Polars and IBIS tests with DB2 URL and connectorX #149

Open windiana42 opened 5 months ago

windiana42 commented 5 months ago

I noticed strange errors with IBIS and Polars on my Linux machine. Strangely, the problem did not show up in CI which also runs Linux. ConnectorX does not officially support DB2, so it is hard to check whether it should work. I disabled the tests for this combination but this issue is to investigate whether ConnectorX really can't work with DB2 or whether there is a fix or workaround:

E           RuntimeError: parse error: relative URL without a base

../../../.cache/pypoetry/virtualenvs/pydiverse-pipedag-Hmb_rarN-py3.10/lib/python3.10/site-packages/connectorx/__init__.py:297: RuntimeError

This is the URL we use: db2+ibm_db://db2inst1:password@localhost:50000/testdb

windiana42 commented 5 months ago

https://github.com/pydiverse/pydiverse.pipedag/pull/148/commits/58c008cd7daccfeffc049eae86af3e72777a7eef