test "tests/table/test_from_py.py" fails on clean repo

[universalmind303]

Describe the bug

running pytest tests/table/test_from_py.py fails on a fresh clone of the repo.

To Reproduce

> git clone https://github.com/Eventual-Inc/Daft.git
> cd Daft
> make build
> source .venv/bin/activate
> pytest tests/table/test_from_py.py

ERROR tests/table/test_from_py.py - TypeError: Argument 'typ' has incorrect type (expected pyarrow.lib.BaseExtensionType, got pyarrow.lib.StructType)

Expected behavior

test runs as expected.

Component(s)

Other

Additional context

full error log

pytest tests/table/test_from_py.py ================================================= test session starts ================================================= platform darwin -- Python 3.11.4, pytest-7.4.3, pluggy-1.5.0 codspeed: 2.2.1 (callgraph: not supported) benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000) rootdir: /Volumes/External/Development/Daft configfile: pyproject.toml plugins: codspeed-2.2.1, cov-4.1.0, benchmark-4.0.0, lazy-fixture-0.6.3, anyio-4.6.2.post1, hypothesis-6.79.2 collected 0 items / 1 error ======================================================= ERRORS ======================================================== ____________________________________ ERROR collecting tests/table/test_from_py.py _____________________________________ .venv/lib/python3.11/site-packages/_pytest/runner.py:341: in from_call result: Optional[TResult] = func() .venv/lib/python3.11/site-packages/_pytest/runner.py:372: in call = CallInfo.from_call(lambda: list(collector.collect()), "collect") .venv/lib/python3.11/site-packages/_pytest/python.py:531: in collect self._inject_setup_module_fixture() .venv/lib/python3.11/site-packages/_pytest/python.py:545: in _inject_setup_module_fixture self.obj, ("setUpModule", "setup_module") .venv/lib/python3.11/site-packages/_pytest/python.py:310: in obj self._obj = obj = self._getobj() .venv/lib/python3.11/site-packages/_pytest/python.py:528: in _getobj return self._importtestmodule() .venv/lib/python3.11/site-packages/_pytest/python.py:617: in _importtestmodule mod = import_path(self.path, mode=importmode, root=self.config.rootpath) .venv/lib/python3.11/site-packages/_pytest/pathlib.py:567: in import_path importlib.import_module(module_name) /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py:126: in import_module return _bootstrap._gcd_import(name[level:], package, level) :1204: in _gcd_import ??? :1176: in _find_and_load ??? :1147: in _find_and_load_unlocked ??? :690: in _load_unlocked ??? .venv/lib/python3.11/site-packages/_pytest/assertion/rewrite.py:186: in exec_module exec(co, module.__dict__) tests/table/test_from_py.py:120: in "tensor": pa.ExtensionArray.from_storage( E TypeError: Argument 'typ' has incorrect type (expected pyarrow.lib.BaseExtensionType, got pyarrow.lib.StructType) =============================================== short test summary info =============================================== ERROR tests/table/test_from_py.py - TypeError: Argument 'typ' has incorrect type (expected pyarrow.lib.BaseExtensionType, got pyarrow.lib.StructType) !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ================================================== 1 error in 0.14s ===================================================

[colin-ho]

This is an issue with the registration of extension types, we merged in a temporary fix here: https://github.com/Eventual-Inc/Daft/pull/3030. I guess maybe we need to run each test one at a time and to make sure? 😢 cc @desmondcheongzx

[desmondcheongzx]

I think we can just reregister extension types at import time then untangle this if we want to remove it again