KamitaniLab / bdpy

Python package for brain decoding analysis (BrainDecoderToolbox2 data format, machine learning analysis, functional MRI)
MIT License
33 stars 22 forks source link

Error around environment setting #51

Closed ganow closed 1 year ago

ganow commented 1 year ago

This package specifies the following third-party modules in setup.py.

https://github.com/KamitaniLab/bdpy/blob/bdc866716aab675b311846528a4a275bb075d0fe/setup.py#L49-L56

However, this package uses other modules which are not explicitly specified. Here is the result of the execution of test codes:

$ pytest .
====================================================== test session starts =======================================================
platform linux -- Python 3.8.16, pytest-7.2.2, pluggy-1.0.0
rootdir: /workspaces/bdpy
plugins: anyio-3.6.2
collected 74 items / 5 errors                                                                                                    

============================================================= ERRORS =============================================================
________________________________________________ ERROR collecting test/test_cv.py ________________________________________________
ImportError while importing test module '/workspaces/bdpy/test/test_cv.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test/test_cv.py:12: in <module>
    from bdpy.ml.crossvalidation import cvindex_groupwise, make_cvindex
bdpy/ml/__init__.py:8: in <module>
    from .learning import Classification, CrossValidation, ModelTraining, ModelTest
bdpy/ml/learning.py:18: in <module>
    from bdpy.dataform import save_array, load_array
bdpy/dataform/__init__.py:7: in <module>
    from .pd import *
bdpy/dataform/pd.py:10: in <module>
    import pandas as pd
E   ModuleNotFoundError: No module named 'pandas'
_________________________________________ ERROR collecting test/test_dataform_sparse.py __________________________________________
ImportError while importing test module '/workspaces/bdpy/test/test_dataform_sparse.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test/test_dataform_sparse.py:8: in <module>
    from bdpy.dataform import load_array, save_array
bdpy/dataform/__init__.py:7: in <module>
    from .pd import *
bdpy/dataform/pd.py:10: in <module>
    import pandas as pd
E   ModuleNotFoundError: No module named 'pandas'
________________________________________________ ERROR collecting test/test_ml.py ________________________________________________
ImportError while importing test module '/workspaces/bdpy/test/test_ml.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test/test_ml.py:12: in <module>
    from bdpy.dataform import load_array
bdpy/dataform/__init__.py:7: in <module>
    from .pd import *
bdpy/dataform/pd.py:10: in <module>
    import pandas as pd
E   ModuleNotFoundError: No module named 'pandas'
_____________________________________________ ERROR collecting test/test_ml_utils.py _____________________________________________
ImportError while importing test module '/workspaces/bdpy/test/test_ml_utils.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test/test_ml_utils.py:8: in <module>
    from bdpy import ml
bdpy/ml/__init__.py:8: in <module>
    from .learning import Classification, CrossValidation, ModelTraining, ModelTest
bdpy/ml/learning.py:18: in <module>
    from bdpy.dataform import save_array, load_array
bdpy/dataform/__init__.py:7: in <module>
    from .pd import *
bdpy/dataform/pd.py:10: in <module>
    import pandas as pd
E   ModuleNotFoundError: No module named 'pandas'
_______________________________________________ ERROR collecting test/test_mri.py ________________________________________________
ImportError while importing test module '/workspaces/bdpy/test/test_mri.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test/test_mri.py:10: in <module>
    import bdpy.mri as bmr
bdpy/mri/__init__.py:7: in <module>
    from .load_epi import load_epi
bdpy/mri/load_epi.py:12: in <module>
    import nipy
E   ModuleNotFoundError: No module named 'nipy'
======================================================== warnings summary ========================================================
bdpy/bdata/bdata.py:761
  /workspaces/bdpy/bdpy/bdata/bdata.py:761: DeprecationWarning: invalid escape sequence \*
    key_esc = re.escape(key).replace('\*', '.*') + '$'  # `fullmatch` is available in Python >= 3.4

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_bdata.py)
    class TestLoader(object):

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_bdata_metadata.py)
    class TestLoader(object):

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_bdata_utils.py)
    class TestLoader(object):

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_distcomp.py)
    class TestLoader(object):

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_feature.py)
    class TestLoader(object):

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_featureselector.py)
    class TestLoader(object):

bdpy/preproc/preprocessor.py:13
  /workspaces/bdpy/bdpy/preproc/preprocessor.py:13: PendingDeprecationWarning: Importing from numpy.matlib is deprecated since 1.19.0. The matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray. 
    from numpy.matlib import repmat

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_preproc.py)
    class TestLoader(object):

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_stats.py)
    class TestLoader(object):

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_util.py)
    class TestLoader(object):

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_util_math.py)
    class TestLoader(object):

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
==================================================== short test summary info =====================================================
ERROR test/test_cv.py
ERROR test/test_dataform_sparse.py
ERROR test/test_ml.py
ERROR test/test_ml_utils.py
ERROR test/test_mri.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 5 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================= 12 warnings, 5 errors in 1.40s =================================================

I think it would be a good idea to add pandas and nipy to the required packages. Thank you.

ganow commented 1 year ago

Also, the test codes depend on another third-party module fastl2lir. I believe the test codes should be able to be executed without specifying any additional packages. The main body of bdpy seems not to depend on fastl2lir, but only on the test code, so it seems this can be solved by rewriting the test code.

============================= test session starts ==============================
platform linux -- Python 3.8.16, pytest-7.2.2, pluggy-1.0.0
rootdir: /workspaces/bdpy
plugins: anyio-3.6.2
collected 86 items / 2 errors

==================================== ERRORS ====================================
_______________________ ERROR collecting test/test_cv.py _______________________
ImportError while importing test module '/workspaces/bdpy/test/test_cv.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test/test_cv.py:15: in <module>
    from fastl2lir import FastL2LiR
E   ModuleNotFoundError: No module named 'fastl2lir'
_______________________ ERROR collecting test/test_ml.py _______________________
ImportError while importing test module '/workspaces/bdpy/test/test_ml.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test/test_ml.py:17: in <module>
    from fastl2lir import FastL2LiR
E   ModuleNotFoundError: No module named 'fastl2lir'
=============================== warnings summary ===============================
../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_bdata.py)
    class TestLoader(object):

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_bdata_metadata.py)
    class TestLoader(object):

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_bdata_utils.py)
    class TestLoader(object):

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_dataform_sparse.py)
    class TestLoader(object):

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_distcomp.py)
    class TestLoader(object):

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_feature.py)
    class TestLoader(object):

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_featureselector.py)
    class TestLoader(object):

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_ml_utils.py)
    class TestLoader(object):

.venv/lib/python3.8/site-packages/nibabel/quaternions.py:31
  /workspaces/bdpy/.venv/lib/python3.8/site-packages/nibabel/quaternions.py:31: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    MAX_FLOAT = np.maximum_sctype(np.float)

.venv/lib/python3.8/site-packages/nibabel/quaternions.py:32
  /workspaces/bdpy/.venv/lib/python3.8/site-packages/nibabel/quaternions.py:32: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    FLOAT_EPS = np.finfo(np.float).eps

.venv/lib/python3.8/site-packages/nipy/core/reference/coordinate_system.py:75
  /workspaces/bdpy/.venv/lib/python3.8/site-packages/nipy/core/reference/coordinate_system.py:75: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    dtype = np.dtype([('x', np.float),

.venv/lib/python3.8/site-packages/nipy/core/reference/coordinate_system.py:76
  /workspaces/bdpy/.venv/lib/python3.8/site-packages/nipy/core/reference/coordinate_system.py:76: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    ('y', np.float),

.venv/lib/python3.8/site-packages/nipy/core/reference/coordinate_system.py:77
  /workspaces/bdpy/.venv/lib/python3.8/site-packages/nipy/core/reference/coordinate_system.py:77: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    ('z', np.float)])

.venv/lib/python3.8/site-packages/nipy/core/reference/coordinate_system.py:82
  /workspaces/bdpy/.venv/lib/python3.8/site-packages/nipy/core/reference/coordinate_system.py:82: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    def __init__(self, coord_names, name='', coord_dtype=np.float):

.venv/lib/python3.8/site-packages/nipy/core/reference/coordinate_system.py:423
  /workspaces/bdpy/.venv/lib/python3.8/site-packages/nipy/core/reference/coordinate_system.py:423: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    def __init__(self, coord_names, name='', coord_dtype=np.float):

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_mri.py)
    class TestLoader(object):

bdpy/preproc/preprocessor.py:13
  /workspaces/bdpy/bdpy/preproc/preprocessor.py:13: PendingDeprecationWarning: Importing from numpy.matlib is deprecated since 1.19.0. The matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray. 
    from numpy.matlib import repmat

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_preproc.py)
    class TestLoader(object):

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_stats.py)
    class TestLoader(object):

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_util.py)
    class TestLoader(object):

../../usr/local/lib/python3.8/unittest/loader.py:66
  /usr/local/lib/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: test/test_util_math.py)
    class TestLoader(object):

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
ERROR test/test_cv.py
ERROR test/test_ml.py
!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!
======================== 21 warnings, 2 errors in 1.40s ========================
ganow commented 1 year ago

I'm trying to solve this issue in #67

For the issue below, now I'm putting fastl2lir as an additional dev dependency as a workaround.

Also, the test codes depend on another third-party module fastl2lir. I believe the test codes should be able to be executed without specifying any additional packages. The main body of bdpy seems not to depend on fastl2lir, but only on the test code, so it seems this can be solved by rewriting the test code.

https://github.com/KamitaniLab/bdpy/blob/8c35aceb6742d41250553cc886606e351a4b5e50/pyproject.toml#L51-L58

ganow commented 1 year ago

solved