msmbuilder / msmbuilder2022

Statistical models for biomolecular dynamics
GNU Lesser General Public License v2.1
37 stars 17 forks source link

Running tests #12

Closed braniii closed 1 year ago

braniii commented 1 year ago

Hi, I am trying to run the tests.

Creating the environment with:

git clone git@github.com:braniii/msmbuilder2022.git
cd msmbuilder2022/

conda create -n msmbuilder -c conda-forge python=3.10
conda activate msmbuilder

python -m pip install -e .
python -m pip install pytest numdifftools hmmlearn nose cython

Running

>>> python runtests.py
Building, see build.log...
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/10/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa:hsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 10.2.1-6' --with-bugurl=file:///usr/share/doc/gcc-10/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-10 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-gcn/usr,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-build-config=bootstrap-lto-lean --enable-link-mutex
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 10.2.1 20210110 (Debian 10.2.1-6) 
C compiler:
Attempting to autodetect OpenMP support... Compiler supports OpenMP
Attempting to autodetect SSE3   support... Compiler supports SSE3
Attempting to autodetect SSE4.1 support... Compiler supports SSE4.1

running build
running build_py
copying msmbuilder/version.py -> build/lib.linux-x86_64-cpython-310/msmbuilder
running build_ext
Traceback (most recent call last):
  File "/home/braniii/Documents/Github/msmbuilder2022/setup.py", line 195, in <module>
    setup(name='msmbuilder2022',
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/setuptools/__init__.py", line 103, in setup
    return distutils.core.setup(**attrs)
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/setuptools/dist.py", line 989, in run_command
    super().run_command(command)
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 131, in run
    self.run_command(cmd_name)
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/setuptools/dist.py", line 989, in run_command
    super().run_command(command)
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
    self.build_extensions()
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
    self._build_extensions_serial()
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/Cython/Distutils/build_ext.py", line 130, in build_extension
    new_ext = cythonize(
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1010, in cythonize
    module_list, module_metadata = create_extension_list(
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 859, in create_extension_list
    kwds = deps.distutils_info(file, aliases, base).values
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 707, in distutils_info
    return (self.transitive_merge(filename, self.distutils_info0, DistutilsInfo.merge)
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 716, in transitive_merge
    return self.transitive_merge_helper(
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 722, in transitive_merge_helper
    deps = extract(node)
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 687, in distutils_info0
    cimports, externs, incdirs = self.cimports_externs_incdirs(filename)
  File "Cython/Utils.py", line 129, in Cython.Utils.cached_method.wrapper
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 574, in cimports_externs_incdirs
    for include in self.included_files(filename):
  File "Cython/Utils.py", line 129, in Cython.Utils.cached_method.wrapper
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 556, in included_files
    include_path = self.context.find_include_file(include, source_file_path=filename)
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/Cython/Compiler/Main.py", line 299, in find_include_file
    error(pos, "'%s' not found" % filename)
  File "/home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/Cython/Compiler/Errors.py", line 178, in error
    raise InternalError(message)
Cython.Compiler.Errors.InternalError: Internal compiler error: 'cy_blas.pyx' not found

Build failed!

Following the gh action, it seems that the module pyhmc is missing the dependency statsmodels

>>> python msmbuilder/tests/data_init.py
>>> python -m pytest msmbuilder/tests/ -v
======================================== test session starts =========================================
platform linux -- Python 3.10.12, pytest-7.4.2, pluggy-1.3.0 -- /home/braniii/anaconda3/envs/msmbuilder/bin/python
cachedir: .pytest_cache
rootdir: /home/braniii/Documents/Github/msmbuilder2022
collected 320 items / 1 error                                                                        

=============================================== ERRORS ===============================================
_____________________ ERROR collecting msmbuilder/tests/test_bayes_ratematrix.py _____________________
ImportError while importing test module '/home/braniii/Documents/Github/msmbuilder2022/msmbuilder/tests/test_bayes_ratematrix.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../anaconda3/envs/msmbuilder/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
msmbuilder/tests/test_bayes_ratematrix.py:7: in <module>
    from pyhmc import hmc
../../../anaconda3/envs/msmbuilder/lib/python3.10/site-packages/pyhmc/__init__.py:28: in <module>
    from .autocorr1 import integrated_autocorr1
../../../anaconda3/envs/msmbuilder/lib/python3.10/site-packages/pyhmc/autocorr1.py:6: in <module>
    from statsmodels.tsa.stattools import acf
E   ModuleNotFoundError: No module named 'statsmodels'
========================================== warnings summary ==========================================
../../../anaconda3/envs/msmbuilder/lib/python3.10/site-packages/mdtraj/geometry/order.py:25
  /home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/mdtraj/geometry/order.py:25: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    from pkg_resources import parse_version

../../../anaconda3/envs/msmbuilder/lib/python3.10/site-packages/pkg_resources/__init__.py:2871
  /home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('sphinxcontrib')`.
  Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
    declare_namespace(pkg)

msmbuilder/utils/io.py:17
  /home/braniii/Documents/Github/msmbuilder2022/msmbuilder/utils/io.py:17: PendingDeprecationWarning: This module might be deprecated in favor of msmbuilder.io
    warnings.warn("This module might be deprecated in favor of msmbuilder.io",

msmbuilder/cluster/apm.py:209
  /home/braniii/Documents/Github/msmbuilder2022/msmbuilder/cluster/apm.py:209: SyntaxWarning: "is" with a literal. Did you mean "=="?
    if self._get_Lagtime(micro_state, macro_state) is 0:

msmbuilder/featurizer/featurizer.py:1006
  /home/braniii/Documents/Github/msmbuilder2022/msmbuilder/featurizer/featurizer.py:1006: DeprecationWarning: invalid escape sequence '\s'
    """Featurizer based on residue-residue distances.

msmbuilder/featurizer/multiseq_featuizer.py:14
  /home/braniii/Documents/Github/msmbuilder2022/msmbuilder/featurizer/multiseq_featuizer.py:14: DeprecationWarning: invalid escape sequence '\s'
    """Featurizer based on residue-residue contacts of an alignment file.

msmbuilder/featurizer/multiseq_featuizer.py:89
  /home/braniii/Documents/Github/msmbuilder2022/msmbuilder/featurizer/multiseq_featuizer.py:89: SyntaxWarning: "is" with a literal. Did you mean "=="?
    if contacts is 'all':

msmbuilder/featurizer/multichain.py:58
  /home/braniii/Documents/Github/msmbuilder2022/msmbuilder/featurizer/multichain.py:58: SyntaxWarning: "is not" with a literal. Did you mean "!="?
    if reference_frame.n_frames is not 1:

msmbuilder/featurizer/multichain.py:204
  /home/braniii/Documents/Github/msmbuilder2022/msmbuilder/featurizer/multichain.py:204: SyntaxWarning: "is not" with a literal. Did you mean "!="?
    if self.binding_pocket is not 'all':

msmbuilder/featurizer/multichain.py:442
  /home/braniii/Documents/Github/msmbuilder2022/msmbuilder/featurizer/multichain.py:442: SyntaxWarning: "is not" with a literal. Did you mean "!="?
    if align_by is not 'custom':

msmbuilder/featurizer/multichain.py:456
  /home/braniii/Documents/Github/msmbuilder2022/msmbuilder/featurizer/multichain.py:456: SyntaxWarning: "is not" with a literal. Did you mean "!="?
    if calculate_for is not 'custom':

msmbuilder/dataset.py:311
  /home/braniii/Documents/Github/msmbuilder2022/msmbuilder/dataset.py:311: DeprecationWarning: invalid escape sequence '\d'
    _ITEM_RE = re.compile('(\d{8}).npy')

msmbuilder/dataset.py:356
  /home/braniii/Documents/Github/msmbuilder2022/msmbuilder/dataset.py:356: DeprecationWarning: invalid escape sequence '\d'
    _ITEM_RE = re.compile('arr_(\d+)')

msmbuilder/dataset.py:34
  /home/braniii/Documents/Github/msmbuilder2022/msmbuilder/dataset.py:34: PendingDeprecationWarning: This module might be deprecated in favor of msmbuilder.io
    warnings.warn("This module might be deprecated in favor of msmbuilder.io",

msmbuilder/decomposition/ktica.py:23
  /home/braniii/Documents/Github/msmbuilder2022/msmbuilder/decomposition/ktica.py:23: DeprecationWarning: invalid escape sequence '\l'
    """Time-structure Independent Componenent Analysis (tICA) using the kernel trick.

msmbuilder/decomposition/tica.py:27
  /home/braniii/Documents/Github/msmbuilder2022/msmbuilder/decomposition/tica.py:27: DeprecationWarning: invalid escape sequence '\l'
    """Time-structure Independent Component Analysis (tICA)

msmbuilder/tests/test_commands_exist.py:136
  msmbuilder/tests/test_commands_exist.py:136: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_all_help_works will be ignored
    @pytest.mark.skipif(True,reason="takes a long time")

msmbuilder/tests/test_commands_exist.py:142
  msmbuilder/tests/test_commands_exist.py:142: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_all_listed will be ignored
    def test_all_listed():

msmbuilder/tests/test_cyblas_wrapper.py:5
  msmbuilder/tests/test_cyblas_wrapper.py:5: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test will be ignored
    def test():

../../../anaconda3/envs/msmbuilder/lib/python3.10/site-packages/nose/importer.py:12
  /home/braniii/anaconda3/envs/msmbuilder/lib/python3.10/site-packages/nose/importer.py:12: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses
    from imp import find_module, load_module, acquire_lock, release_lock

msmbuilder/tests/test_ghmm.py:85
  msmbuilder/tests/test_ghmm.py:85: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_2_state will be ignored
    def test_2_state():

msmbuilder/tests/test_ghmm.py:113
  msmbuilder/tests/test_ghmm.py:113: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_3_state will be ignored
    def test_3_state():

msmbuilder/tests/test_libdistance.py:28
  msmbuilder/tests/test_libdistance.py:28: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_assign_nearest_double_float_1 will be ignored
    def test_assign_nearest_double_float_1():

msmbuilder/tests/test_libdistance.py:53
  msmbuilder/tests/test_libdistance.py:53: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_assign_nearest_float_double_2 will be ignored
    def test_assign_nearest_float_double_2():

msmbuilder/tests/test_libdistance.py:115
  msmbuilder/tests/test_libdistance.py:115: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_cdist_double_float_1 will be ignored
    def test_cdist_double_float_1():

msmbuilder/tests/test_libdistance.py:127
  msmbuilder/tests/test_libdistance.py:127: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_pdist_double_float_1 will be ignored
    def test_pdist_double_float_1():

msmbuilder/tests/test_libdistance.py:139
  msmbuilder/tests/test_libdistance.py:139: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_pdist_double_float_2 will be ignored
    def test_pdist_double_float_2():

msmbuilder/tests/test_libdistance.py:185
  msmbuilder/tests/test_libdistance.py:185: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_dist_double_float_1 will be ignored
    def test_dist_double_float_1():

msmbuilder/tests/test_libdistance.py:196
  msmbuilder/tests/test_libdistance.py:196: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_dist_double_float_2 will be ignored
    def test_dist_double_float_2():

msmbuilder/tests/test_ratematrix.py:358
  msmbuilder/tests/test_ratematrix.py:358: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_uncertainties_backward will be ignored
    def test_uncertainties_backward():

msmbuilder/tests/test_workflows.py:60
  msmbuilder/tests/test_workflows.py:60: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_workflows will be ignored
    def test_workflows():

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
====================================== short test summary info =======================================
ERROR msmbuilder/tests/test_bayes_ratematrix.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=================================== 31 warnings, 1 error in 41.90s ===================================

Does anyone has an idea why it fails on my machine, but runs on the gh action?

bojunliu0818 commented 1 year ago

Thanks for raising the issue! Yes, statsmodels is an additional required package for pyhmc. Previously I used "conda install -c conda-forge numdifftools", and statsmodels would be installed simultaneously. If you used pip install, then you may need to manually install statsmodels.

braniii commented 1 year ago

Installing statsmodels most tests pass (296 passed, 16 skipped, 13xfailed) just like in the CI. Thx