Closed jpivarski closed 1 year ago
I just tried this with new VMs on AWS, which have no history of ever having Python installed (other than system Python). I tried installing awkward
and dask-awkward
three ways:
In all cases, tests/test_2682_custom_pickler.py
passed without dask-awkward
installed and failed with dask-awkward
installed.
This is very consistent, and now on a platform with no history. I have no idea how our CI manages to not have a problem with this.
The reason our CI doesn't catch it is because dask-awkward doesn't get installed in our CI.
The issue has been reproduced in CI:
https://github.com/scikit-hep/awkward/actions/runs/6411516875/job/17407129086?pr=2739
=================================== FAILURES ===================================
_____________________________ test_default_pickler _____________________________
def test_default_pickler():
> assert _pickle_complex_array_and_return_form_impl() == ak.forms.from_dict(
{"class": "ListOffsetArray", "offsets": "i64", "content": "int64"}
)
E AssertionError: assert ListForm('i64', 'i64', NumpyForm('int64')) == ListOffsetForm('i64', NumpyForm('int64'))
E + where ListForm('i64', 'i64', NumpyForm('int64')) = _pickle_complex_array_and_return_form_impl()
E + and ListOffsetForm('i64', NumpyForm('int64')) = <function from_dict at 0x7fdfc7fabec0>({'class': 'ListOffsetArray', 'content': 'int64', 'offsets': 'i64'})
E + where <function from_dict at 0x7fdfc7fabec0> = <module 'awkward.forms' from '/home/runner/micromamba/envs/awkward/lib/python3.11/site-packages/awkward/forms/__init__.py'>.from_dict
E + where <module 'awkward.forms' from '/home/runner/micromamba/envs/awkward/lib/python3.11/site-packages/awkward/forms/__init__.py'> = ak.forms
tests/test_2682_custom_pickler.py:44: AssertionError
______________________________ test_noop_pickler _______________________________
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/runner/micromamba/envs/awkward/lib/python3.11/concurrent/futures/process.py", line 256, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/awkward/awkward/tests/test_2682_custom_pickler.py", line 26, in _pickle_complex_array_and_return_form_impl
return pickle.loads(pickle.dumps(array)).layout.form
^^^^^^^^^^^^^^^^^^^
self = None
def __get_result(self):
if self._exception:
try:
> raise self._exception
E RuntimeError: Encountered multiple Awkward pickle reducers under the `awkward.pickle.reduce` entrypoint
self = None
../../../micromamba/envs/awkward/lib/python3.11/concurrent/futures/_base.py:401: RuntimeError
During handling of the above exception, another exception occurred:
tmp_path = PosixPath('/tmp/pytest-of-runner/pytest-0/test_malformed_pickler0')
def test_malformed_pickler(tmp_path):
> with pytest.raises(RuntimeError, match=r"malformed pickler!"):
E AssertionError: Regex pattern did not match.
E Regex: 'malformed pickler!'
E Input: 'Encountered multiple Awkward pickle reducers under the `awkward.pickle.reduce` entrypoint'
tmp_path = PosixPath('/tmp/pytest-of-runner/pytest-0/test_malformed_pickler0')
tests/test_2682_custom_pickler.py:87: AssertionError
=========================== short test summary info ============================
SKIPPED [1] tests/test_0115_generic_reducer_operation.py:1209: I can't think of a canonical UnionArray (non-mergeable contents) that can be used in a reducer
SKIPPED [1] tests/test_0401_add_categorical_type_for_arrow_dictionary.py:400: Fix issues for categorical type
SKIPPED [1] tests/test_0652_tests_of_complex_numbers.py:218: Remember to implement sorting for complex numbers.
SKIPPED [1] tests/test_1072_sort.py:714: I can't think of a canonical UnionArray (non-mergeable contents) that can be used in sorting
SKIPPED [1] tests/test_1072_sort.py:739: I can't think of a canonical UnionArray (non-mergeable contents) that can be used in sorting
SKIPPED [1] tests/test_1300_awkward_to_cpp_converter_with_cling.py:345: ROOT was compiled without C++17 support
SKIPPED [1] tests/test_1300_awkward_to_cpp_converter_with_cling.py:408: ROOT was compiled without C++17 support
SKIPPED [1] tests/test_1300_awkward_to_cpp_converter_with_cling.py:467: ROOT was compiled without C++17 support
SKIPPED [1] tests/test_1300_awkward_to_cpp_converter_with_cling.py:788: ROOT was compiled without C++17 support
SKIPPED [4] tests/test_1440_start_v2_to_parquet.py:223: Categorical arrays can't roundtrip through Parquet due to ARROW-14525
SKIPPED [2] tests/test_1613_generator_tolayout_records.py:145: ROOT was compiled without C++17 support
SKIPPED [2] tests/test_1613_generator_tolayout_records.py:182: ROOT was compiled without C++17 support
SKIPPED [2] tests/test_1613_generator_tolayout_records.py:202: ROOT was compiled without C++17 support
SKIPPED [1] tests/test_1613_generator_tolayout_records.py:291: the test needs an external data file: see the comments
SKIPPED [1] tests/test_1672_broadcast_parameters.py:9: string broadcasting is broken
SKIPPED [1] tests/test_1672_broadcast_parameters.py:23: string broadcasting is broken
SKIPPED [1] tests/test_1672_broadcast_parameters.py:37: string broadcasting is broken
SKIPPED [1] tests/test_1672_broadcast_parameters.py:55: string broadcasting is broken
SKIPPED [1] tests/test_1764_jax_jacobian.py:14: Jacobian support not implemented
SKIPPED [1] tests/test_2306_cppyy_jit.py:48: Awkward Array can only work with cppyy 3.0.1 or later.
SKIPPED [1] tests/test_2306_cppyy_jit.py:83: Awkward Array can only work with cppyy 3.0.1 or later.
SKIPPED [1] tests/test_2327_array_interface.py:10: could not import 'cupy': No module named 'cupy'
SKIPPED [1] tests/test_2649_dlpack_support.py:18: could not import 'cupy': No module named 'cupy'
============ 4 failed, 2490 passed, 29 skipped in 239.86s (0:03:59) ============
Ah, this is a testing problem rather than a bug per-se. I think what's happening is that dask-awkward registers its own pickler, in addition to the pickler that is registered by the test itself. As such, we find two picklers, and fail. I think the solution should involve hiding the dask-awkward pickler from our test suite, if we can.
Version of Awkward Array
2.4.4 and HEAD
Description and code to reproduce
I've been tracking down my inability to run
tests/test_2682_custom_pickler.py
locally. It's pretty consistent about working when I don't have dask-awkward installed and not working when I do have dask-awkward installed.Notes
I haven't seen this issue before, but it's the first time I'm running in Python 3.10 environments. I don't know if that's related. (Or maybe I've never tested it before because I didn't have dask-awkward installed between the time when the custom picklers were added and now.)
Also, I can't say why CI isn't seeing this. Perhaps it has something to do with conda.
Also, I verified that I don't have any self-installed libraries outside of the conda environment. In particular,
shows nothing outside of my environment and dask-awkward's contribution within it, if dask-awkward has been installed.
Clean-slate test
I created a new conda environment with these initial packages:
First, I install
awkward
from conda-forge:The tests pass.
Then, I install
dask-awkward
from conda-forge:The tests fail.
Then I remove them both.
Next, I install
awkward
from PyPI:The tests pass.
Then I install
dask-awkward
from PyPI:The tests fail.
Next, I remove them both.
Now I install
awkward
from source (main
), using the nox,awkward-cpp
compilation, install-in-place procedure.The tests pass.
Then I install
dask-awkward
from source (main
).The tests fail.
Error messages
When the tests fail, they fail like this:
I'm going to see if this can be captured in a GitHub Action.