Issues with running cryolo

[douweschulte]

We have cryolo1.9.1_cu11 installed on a shared workstation with the help of SBGrid. The workstation is running Ubuntu 20. When running the program for the first time the following error popped up:

(base) martas@Snijderlab-GPU:~/221207_IgM$ cryolo_gui.pyTraceback (most recent call last):  File "/programs/x86_64-linux/cryolo/1.9.1_cu11/cryolo/bin/cryolo_gui.py", line 8, in     sys.exit(_main_())  File "/programs/x86_64-linux/cryolo/1.9.1_cu11/cryolo_extlib/miniconda3-py310_22.11-aego/lib/python3.8/site-packages/cryolo/cryolo_main.py", line 450, in _main_    PARSER = create_parser()  File "/programs/x86_64-linux/cryolo/1.9.1_cu11/cryolo_extlib/miniconda3-py310_22.11-aego/lib/python3.8/site-packages/cryolo/cryolo_main.py", line 307, in create_parser    from cryolo.train import create_parser as create_train_parser  File "/programs/x86_64-linux/cryolo/1.9.1_cu11/cryolo_extlib/miniconda3-py310_22.11-aego/lib/python3.8/site-packages/cryolo/train.py", line 39, in     import cryolo.preprocessing as preprocess  File "/programs/x86_64-linux/cryolo/1.9.1_cu11/cryolo_extlib/miniconda3-py310_22.11-aego/lib/python3.8/site-packages/cryolo/preprocessing.py", line 21, in     import cryolo.CoordsIO as CoordsIO  File "/programs/x86_64-linux/cryolo/1.9.1_cu11/cryolo_extlib/miniconda3-py310_22.11-aego/lib/python3.8/site-packages/cryolo/CoordsIO.py", line 23, in     import cryolo.utils as utils  File "/programs/x86_64-linux/cryolo/1.9.1_cu11/cryolo_extlib/miniconda3-py310_22.11-aego/lib/python3.8/site-packages/cryolo/utils.py", line 27, in     from sklearn import mixture  File "/programs/x86_64-linux/cryolo/1.9.1_cu11/cryolo_extlib/miniconda3-py310_22.11-aego/lib/python3.8/site-packages/sklearn/mixture/__init__.py", line 5, in     from ._gaussian_mixture import GaussianMixture  File "/programs/x86_64-linux/cryolo/1.9.1_cu11/cryolo_extlib/miniconda3-py310_22.11-aego/lib/python3.8/site-packages/sklearn/mixture/_gaussian_mixture.py", line 11, in     from ._base import BaseMixture, _check_shape  File "/programs/x86_64-linux/cryolo/1.9.1_cu11/cryolo_extlib/miniconda3-py310_22.11-aego/lib/python3.8/site-packages/sklearn/mixture/_base.py", line 14, in     from .. import cluster  File "/programs/x86_64-linux/cryolo/1.9.1_cu11/cryolo_extlib/miniconda3-py310_22.11-aego/lib/python3.8/site-packages/sklearn/cluster/__init__.py", line 6, in     from ._spectral import spectral_clustering, SpectralClustering  File "/programs/x86_64-linux/cryolo/1.9.1_cu11/cryolo_extlib/miniconda3-py310_22.11-aego/lib/python3.8/site-packages/sklearn/cluster/_spectral.py", line 16, in     from ..neighbors import kneighbors_graph, NearestNeighbors  File "/programs/x86_64-linux/cryolo/1.9.1_cu11/cryolo_extlib/miniconda3-py310_22.11-aego/lib/python3.8/site-packages/sklearn/neighbors/__init__.py", line 17, in     from ._nca import NeighborhoodComponentsAnalysis  File "/programs/x86_64-linux/cryolo/1.9.1_cu11/cryolo_extlib/miniconda3-py310_22.11-aego/lib/python3.8/site-packages/sklearn/neighbors/_nca.py", line 22, in     from ..decomposition import PCA  File "/programs/x86_64-linux/cryolo/1.9.1_cu11/cryolo_extlib/miniconda3-py310_22.11-aego/lib/python3.8/site-packages/sklearn/decomposition/__init__.py", line 17, in     from .dict_learning import dict_learning  File "/programs/x86_64-linux/cryolo/1.9.1_cu11/cryolo_extlib/miniconda3-py310_22.11-aego/lib/python3.8/site-packages/sklearn/decomposition/dict_learning.py", line 5, in     from . import _dict_learning  # type: ignore  File "/programs/x86_64-linux/cryolo/1.9.1_cu11/cryolo_extlib/miniconda3-py310_22.11-aego/lib/python3.8/site-packages/sklearn/decomposition/_dict_learning.py", line 21, in     from ..linear_model import Lasso, orthogonal_mp_gram, LassoLars, Lars  File "/programs/x86_64-linux/cryolo/1.9.1_cu11/cryolo_extlib/miniconda3-py310_22.11-aego/lib/python3.8/site-packages/sklearn/linear_model/__init__.py", line 11, in     from ._least_angle import (Lars, LassoLars, lars_path, lars_path_gram, LarsCV,  File "/programs/x86_64-linux/cryolo/1.9.1_cu11/cryolo_extlib/miniconda3-py310_22.11-aego/lib/python3.8/site-packages/sklearn/linear_model/_least_angle.py", line 34, in     method='lar', copy_X=True, eps=np.finfo(np.float).eps,  File "/storage-hdd/martas/.local/lib/python3.8/site-packages/numpy/__init__.py", line 284, in __getattr__    raise AttributeError("module {!r} has no attribute "AttributeError: module 'numpy' has no attribute 'float'

What should we do to fix the issue?

[thorstenwagner]

Hm, look strange for me. Can you show me your environment?

conda activate cryolo
conda list >> conda.txt
pip freeze >> pip.txt

[thorstenwagner]

Can you provide any update on this?

[douweschulte]

I am terribly sorry for the late reply. We tried running 1.8.4_cu11 with the following result:

[10:16] Siborova, M. (Marta) Traceback (most recent call last):   File "/programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo/bin/cryolo_gui.py", line 8, in     sys.exit(_main_())   File "/programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo_extlib/miniconda3-py38_4.8.2-whi7/lib/python3.8/site-packages/cryolo/cryolo_main.py", line 443, in _main_     PARSER = create_parser()   File "/programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo_extlib/miniconda3-py38_4.8.2-whi7/lib/python3.8/site-packages/cryolo/cryolo_main.py", line 311, in create_parser     from cryolo.train import create_parser as create_train_parser   File "/programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo_extlib/miniconda3-py38_4.8.2-whi7/lib/python3.8/site-packages/cryolo/train.py", line 37, in     import cryolo.preprocessing as preprocess   File "/programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo_extlib/miniconda3-py38_4.8.2-whi7/lib/python3.8/site-packages/cryolo/preprocessing.py", line 21, in     import cryolo.CoordsIO as CoordsIO   File "/programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo_extlib/miniconda3-py38_4.8.2-whi7/lib/python3.8/site-packages/cryolo/CoordsIO.py", line 21, in     import cryolo.utils as utils   File "/programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo_extlib/miniconda3-py38_4.8.2-whi7/lib/python3.8/site-packages/cryolo/utils.py", line 27, in     from sklearn import mixture   File "/programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo_extlib/miniconda3-py38_4.8.2-whi7/lib/python3.8/site-packages/sklearn/mixture/__init__.py", line 5, in     from ._gaussian_mixture import GaussianMixture   File "/programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo_extlib/miniconda3-py38_4.8.2-whi7/lib/python3.8/site-packages/sklearn/mixture/_gaussian_mixture.py", line 11, in     from ._base import BaseMixture, _check_shape   File "/programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo_extlib/miniconda3-py38_4.8.2-whi7/lib/python3.8/site-packages/sklearn/mixture/_base.py", line 14, in     from .. import cluster   File "/programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo_extlib/miniconda3-py38_4.8.2-whi7/lib/python3.8/site-packages/sklearn/cluster/__init__.py", line 6, in     from ._spectral import spectral_clustering, SpectralClustering   File "/programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo_extlib/miniconda3-py38_4.8.2-whi7/lib/python3.8/site-packages/sklearn/cluster/_spectral.py", line 16, in     from ..neighbors import kneighbors_graph, NearestNeighbors   File "/programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo_extlib/miniconda3-py38_4.8.2-whi7/lib/python3.8/site-packages/sklearn/neighbors/__init__.py", line 17, in     from ._nca import NeighborhoodComponentsAnalysis   File "/programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo_extlib/miniconda3-py38_4.8.2-whi7/lib/python3.8/site-packages/sklearn/neighbors/_nca.py", line 22, in     from ..decomposition import PCA   File "/programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo_extlib/miniconda3-py38_4.8.2-whi7/lib/python3.8/site-packages/sklearn/decomposition/__init__.py", line 17, in     from .dict_learning import dict_learning   File "/programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo_extlib/miniconda3-py38_4.8.2-whi7/lib/python3.8/site-packages/sklearn/decomposition/dict_learning.py", line 5, in     from . import _dict_learning  # type: ignore   File "/programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo_extlib/miniconda3-py38_4.8.2-whi7/lib/python3.8/site-packages/sklearn/decomposition/_dict_learning.py", line 21, in     from ..linear_model import Lasso, orthogonal_mp_gram, LassoLars, Lars   File "/programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo_extlib/miniconda3-py38_4.8.2-whi7/lib/python3.8/site-packages/sklearn/linear_model/__init__.py", line 11, in     from ._least_angle import (Lars, LassoLars, lars_path, lars_path_gram, LarsCV,   File "/programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo_extlib/miniconda3-py38_4.8.2-whi7/lib/python3.8/site-packages/sklearn/linear_model/_least_angle.py", line 34, in     method='lar', copy_X=True, eps=np.finfo(np.float).eps,   File "/storage-hdd/martas/.local/lib/python3.8/site-packages/numpy/__init__.py", line 284, in __getattr__     raise AttributeError("module {!r} has no attribute " AttributeError: module 'numpy' has no attribute 'float' awk: /programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo/lib/./libtinfo.so.6: no version information available (required by /programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo/lib/libreadline.so.8) awk: /programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo/lib/./libtinfo.so.6: no version information available (required by /programs/x86_64-linux/cryolo/1.8.4_cu11/cryolo/lib/libreadline.so.8)

And with regards to you question, SBGrid installs it very opaquely so there is no conda environment that I can find to list the packages from.

[douweschulte]

But installing separately seems to work, at least it launches in that way. So for now this seems like a good fix. If however you think this could be an error in the setup of cryolo in SBGrid, I will contact them to try and get this sorted for other users.

[thorstenwagner]

That's strange because I know that SBGrid runs crYOLO since a long time. Maybe @jasonkey can help us?

[douweschulte]

I will make it even more strange for you, it turns out cryolo is running fine both through the direct anaconda install and sbgrid on one account, while another account always creates this error for both installs even though they share the same anaconda and sbgrid binaries. Is there some way local settings influence cryolo?

[thorstenwagner]

I don't think so, maybe one user changed $PATH? Can you check $PATH for both of them? They should be the same, right?

[douweschulte]

Yes they run the same cryolo binary, $PATH is hard in this case as both Anaconda and SBGrid find the binaries separate from the normal PATH system.

[jasonkey]

Jason from SBGrid here -

It looks like the user in this case is importing a local version of numpy from their home directory. That version is incompatible with cryolo:

File "/storage-hdd/martas/.local/lib/python3.8/site-packages/numpy/__init__.py", line 284, in __getattr__    raise AttributeError("module {!r} has no attribute "AttributeError: module 'numpy' has no attribute 'float' 

If the application is being run in a standard manner in the SBGrid environment, that shouldn't be possible. So that's either a bug on our end or the environment is not set up properly for the user. Either way, we can definitely fix this.

@douweschulte can you send the output of the sbinfo command to bugs@sbgrid.org from both the system that is working and the system that is not working? From there we should be able to sort this out.

[douweschulte]

I just sent it to you, thanks for helping!

[douweschulte]

Sorry I forgot to give an update here, we have been discussion the issue with @jasonkey and it turned out that an additional local python install managed to interfere with the sbgrid setup. So this issue is not related to cryolo.