search

xarray-contrib / xarray-simlab

Xarray extension and framework for computer model simulations
http://xarray-simlab.readthedocs.io
BSD 3-Clause "New" or "Revised" License
73 stars 9 forks source link

One test fails #192

Open yurivict opened 1 year ago

yurivict commented 1 year ago 
=============================================================================================== FAILURES ================================================================================================
______________________________________________________________________________ TestXarraySimulationDriver.test_multi_index ______________________________________________________________________________

self = <xsimlab.tests.test_drivers.TestXarraySimulationDriver object at 0x947e20880>
in_dataset = <xarray.Dataset>
Dimensions:                 (clock: 5, out: 3, dummy: 2)
Coordinates:
  * clock                   (cl...dummy) int64 0 0
    b                       (dummy) int64 1 2
Attributes:
    __xsimlab_output_vars__:  profile__u_opp
model = <xsimlab.Model (4 processes, 3 inputs)>
init_profile
    n_points     [in] nb. of profile points
roll
    shift        [in] shift profile by a nb. of points
add
    offset       [in] () or ('x',) offset added to profile u
profile

    def test_multi_index(self, in_dataset, model):
        # just check that multi-index pass through model run (reset -> zarr -> rebuilt)
        midx = pd.MultiIndex.from_tuples([(0, 1), (0, 2)], names=["a", "b"])

        in_dataset["dummy"] = ("dummy", midx)

        driver = XarraySimulationDriver(in_dataset, model)
>       driver.run_model()

xsimlab/tests/test_drivers.py:123: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
xsimlab/drivers.py:470: in run_model
    self.store.write_input_xr_dataset()
xsimlab/stores.py:187: in write_input_xr_dataset
    ds.to_zarr(self.zgroup.store, group=self.zgroup.path, mode="a")
/usr/local/lib/python3.9/site-packages/xarray/core/dataset.py:2060: in to_zarr
    return to_zarr(  # type: ignore
/usr/local/lib/python3.9/site-packages/xarray/backends/api.py:1637: in to_zarr
    dump_to_store(dataset, zstore, writer, encoding=encoding)
/usr/local/lib/python3.9/site-packages/xarray/backends/api.py:1257: in dump_to_store
    store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims)
/usr/local/lib/python3.9/site-packages/xarray/backends/zarr.py:537: in store
    variables_encoded, attributes = self.encode(
/usr/local/lib/python3.9/site-packages/xarray/backends/common.py:202: in encode
    variables = {k: self.encode_variable(v) for k, v in variables.items()}
/usr/local/lib/python3.9/site-packages/xarray/backends/common.py:202: in <dictcomp>
    variables = {k: self.encode_variable(v) for k, v in variables.items()}
/usr/local/lib/python3.9/site-packages/xarray/backends/zarr.py:496: in encode_variable
    variable = encode_zarr_variable(variable)
/usr/local/lib/python3.9/site-packages/xarray/backends/zarr.py:285: in encode_zarr_variable
    var = conventions.encode_cf_variable(var, name=name)
/usr/local/lib/python3.9/site-packages/xarray/conventions.py:264: in encode_cf_variable
    ensure_not_multiindex(var, name=name)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

var = <xarray.IndexVariable 'dummy' (dummy: 2)>
array([(0, 1), (0, 2)], dtype=object), name = None

    def ensure_not_multiindex(var, name=None):
        if isinstance(var, IndexVariable) and isinstance(var.to_index(), pd.MultiIndex):
>           raise NotImplementedError(
                "variable {!r} is a MultiIndex, which cannot yet be "
                "serialized to netCDF files "
                "(https://github.com/pydata/xarray/issues/1077). Use "
                "reset_index() to convert MultiIndex levels into coordinate "
                "variables instead.".format(name)
            )
E           NotImplementedError: variable None is a MultiIndex, which cannot yet be serialized to netCDF files (https://github.com/pydata/xarray/issues/1077). Use reset_index() to convert MultiIndex levels into coordinate variables instead.

/usr/local/lib/python3.9/site-packages/xarray/conventions.py:179: NotImplementedError
=========================================================================================== warnings summary ============================================================================================
xsimlab/tests/test_xr_accessor.py:26
  /usr/ports/devel/py-xarray-simlab/work-py39/xarray-simlab-0.5.0-5-g33a4c83/xsimlab/tests/test_xr_accessor.py:26: PytestDeprecationWarning: @pytest.yield_fixture is deprecated.
  Use @pytest.fixture instead; they are the same.
    @pytest.yield_fixture(scope="module", params=use_dask_schedulers)

../../../../../local/lib/python3.9/site-packages/pytest_freezegun.py:17: 606 warnings
  /usr/local/lib/python3.9/site-packages/pytest_freezegun.py:17: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    if LooseVersion(pytest.__version__) < LooseVersion('3.6.0'):

xsimlab/tests/test_utils.py::TestAttrMapping::test_eq
  /usr/ports/devel/py-xarray-simlab/work-py39/xarray-simlab-0.5.0-5-g33a4c83/xsimlab/tests/test_utils.py:110: DeprecationWarning: NotImplemented should not be used in a boolean context
    assert bool(attr_mapping.__eq__(2))  # bool(NotImplemented) == True

xsimlab/tests/test_xr_accessor.py::TestSimlabAccessor::test_run_batch_dim[distributed-False-dims1-data1-clock]
xsimlab/tests/test_xr_accessor.py::TestSimlabAccessor::test_run_batch_dim[distributed-threads-False-dims1-data1-clock]
  /usr/local/lib/python3.9/site-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first
    self.make_current()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================================================================================== short test summary info ========================================================================================
SKIPPED [1] xsimlab/tests/test_dot.py:7: could not import 'graphviz': No module named 'graphviz'
SKIPPED [1] xsimlab/tests/test_model.py:298: could not import 'graphviz': No module named 'graphviz'
SKIPPED [4] xsimlab/tests/test_monitoring.py:9: requires tqdm
SKIPPED [2] xsimlab/tests/test_monitoring.py:43: requires tqdm
SKIPPED [1] xsimlab/tests/test_monitoring.py:56: requires tqdm
SKIPPED [1] xsimlab/tests/test_monitoring.py:37: requires tqdm
SKIPPED [2] xsimlab/tests/test_monitoring.py:26: requires tqdm
SKIPPED [1] xsimlab/tests/test_monitoring.py:65: requires tqdm
SKIPPED [1] xsimlab/tests/test_monitoring.py:85: requires tqdm
SKIPPED [1] xsimlab/tests/test_monitoring.py:76: requires tqdm
======================================================================= 1 failed, 288 passed, 15 skipped, 610 warnings in 52.52s ========================================================================

rev.0.5.0-5-g33a4c83 Python-3.9 FreeBSD 13.1

benbovy commented 1 year ago

Thanks for the report @yurivict. Yes, passing multi-index through a simulation is broken in recent Xarray releases, in which there has been a major refactor regarding (multi-)indexes. This needs to be fixed, but in the meantime it is recommended to manually reset multi-indexes and set it back before/after executing .xsimlab.run().