pandas 0.1.0 breaks lots of tests

[turbach]

Anaconda cloud has begun distributing 0.1.0 and the pandas requirement in fitgrid is unpinned at the moment

May not be too bad ... at least one cause is pandas.core.index is deprecated so asserts are now failing.

Checking ...

[turbach]

Fixing fitgrid won't help given the current pins ... pandas 0.1.0 drops the DataFrame.from_items() method and rpy2 errors when fitting LMER.

Tracking this down will take more digging and version wrangling, I don't have the time at present.

Short term fix is to pin pandas < 0.1.0 for now ...

(test_mkconda.0.0.9) [turbach@mkgpu1 fitgrid]$ python -m pytest -x
============================= test session starts ==============================
platform linux -- Python 3.6.10, pytest-5.3.2, py-1.8.1, pluggy-0.13.1
rootdir: /mnt/cube/home/turbach/TPU_Projects/fitgrid
collected 203 items                                                            

tests/test_epochs.py ......                                              [  2%]
tests/test_fake_data.py .                                                [  3%]
tests/test_fitgrid.py ............F

=================================== FAILURES ===================================
__________________________ test__save_load_grid_lmer ___________________________

tpath = PosixPath('/mnt/cube/home/turbach/TPU_Projects/fitgrid/tests')

    def test__save_load_grid_lmer(tpath):

        epochs = fitgrid.generate(n_samples=2, n_channels=1)
>       grid = fitgrid.lmer(epochs, RHS='continuous + (continuous | categorical)')

tests/test_fitgrid.py:254: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
fitgrid/models.py:273: in lmer
    epochs, function, channels=LHS, parallel=parallel, n_cores=n_cores
fitgrid/models.py:116: in _run_model
    grid = pd.concat(results, axis=1).T
/home/turbach/.conda/envs/test_mkconda.0.0.9/lib/python3.6/site-packages/pandas/core/reshape/concat.py:281: in concat
    sort=sort,
/home/turbach/.conda/envs/test_mkconda.0.0.9/lib/python3.6/site-packages/pandas/core/reshape/concat.py:326: in __init__
    objs = list(objs)
fitgrid/models.py:45: in process_key_and_group
    results = {channel: function(group, channel) for channel in channels}
fitgrid/models.py:45: in <dictcomp>
    results = {channel: function(group, channel) for channel in channels}
fitgrid/models.py:183: in lmer_single
    REML=REML,
/home/turbach/.conda/envs/test_mkconda.0.0.9/lib/python3.6/site-packages/pymer4/models.py:276: in fit
    grps = pandas2ri.ri2py(r_grps)
/home/turbach/.conda/envs/test_mkconda.0.0.9/lib/python3.6/functools.py:807: in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

obj = R object with classes: ('data.frame',) mapped to:
<DataFrame - Python:0x7f98318c9a48 / R:0x5635d241cd88>
[FloatVector]...'>
  R object with classes: ('numeric',) mapped to:
<FloatVector - Python:0x7f98303dbe88 / R:0x5635d241e368>
[2.000000]

    @ri2py.register(DataFrame)
    def ri2py_dataframe(obj):
        items = tuple((k, ri2py(v)) for k, v in obj.items())
>       res = PandasDataFrame.from_items(items)
E       AttributeError: type object 'DataFrame' has no attribute 'from_items'

/home/turbach/.conda/envs/test_mkconda.0.0.9/lib/python3.6/site-packages/rpy2/robjects/pandas2ri.py:191: AttributeError
----------------------------- Captured stderr call -----------------------------
  0%|                                                    | 0/2 [00:00<?, ?it/s]
=============================== warnings summary ===============================
tests/test_fitgrid.py::test__save_load_grid_lm
  /home/turbach/.conda/envs/test_mkconda.0.0.9/lib/python3.6/site-packages/rpy2/robjects/pandas2ri.py:17: FutureWarning: pandas.core.index is deprecated and will be removed in a future version.  The public classes are available in the top-level namespace.
    from pandas.core.index import Index as PandasIndex

tests/test_fitgrid.py::test__save_load_grid_lm
  /home/turbach/.conda/envs/test_mkconda.0.0.9/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject
    return f(*args, **kwds)

-- Docs: https://docs.pytest.org/en/latest/warnings.html
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
================== 1 failed, 19 passed, 2 warnings in 25.83s ===================
(test_mkconda.0.0.9) [turbach@mkgpu1 fitgrid]$

[turbach]

closed by https://github.com/kutaslab/fitgrid/commit/48554ef80415cfb10370a99d7e666cc7c123eeda

[andportnoy]

We are getting rpy2 2.9.4 from Anaconda, but the current version is 3.2.4, which doesn't use this deprecated method. One option is to ping them again so they update the main channel. Another option is to start using conda forge, which has the later versions.

Pymer4 has also been updated in the meantime.

So a longterm solution involves refreshing pymer4, updating to more recent rpy2, then unpinning pandas and fixing any remaining problems. Will try to get to this at some point.