search

bpowell122 / tomodrgn

Neural networks to analyze structural heterogeneity in cryo-electron sub-tomograms
GNU General Public License v3.0
19 stars 1 forks source link

Installation issues #4

Open rdrighetto opened 1 year ago

rdrighetto commented 1 year ago

Hi,

I followed the repo instructions for installing tomoDRGN and encountered a few errors:

First, I was not able to install pytorch-gpu from the pytorch channel, it gave the following error:

$ conda install "pytorch-gpu>=1.11.0" "cudatoolkit>=11.0" -c pytorch                
Collecting package metadata (current_repodata.json): done                                                                                                                 
Solving environment: failed with initial frozen solve. Retrying with flexible solve.                                                                                      
Collecting package metadata (repodata.json): done                                                                                                                         
Solving environment: failed with initial frozen solve. Retrying with flexible solve.                                                                                      

PackagesNotFoundError: The following packages are not available from current channels:                                                                                    

  - pytorch-gpu[version='>=1.11.0']                                                                                                                                       

Current channels:                                                                                                                                                         

  - https://conda.anaconda.org/pytorch/linux-64                                                                                                                           
  - https://conda.anaconda.org/pytorch/noarch                                                                                                                             
  - https://repo.anaconda.com/pkgs/main/linux-64                                                                                                                          
  - https://repo.anaconda.com/pkgs/main/noarch                                                                                                                            
  - https://repo.anaconda.com/pkgs/r/linux-64                                                                                                                             
  - https://repo.anaconda.com/pkgs/r/noarch                                                                                                                               

To search for alternate channels that may provide the conda package you're                                                                                                
looking for, navigate to                                                                                                                                                  

    https://anaconda.org                                                                                                                                                  

and use the search bar at the top of the page.

However, installing from the conda-forge channel succeeded: $ conda install "pytorch-gpu>=1.11.0" "cudatoolkit>=11.0" -c conda-forge and the rest of the installation commands did succeed as well, so I just leave it here for a possible update of the instructions and future reference.

Second, I tried to run the suggested tests, but it failed at the first test:

$ python ./quicktest.py
------  SETUP  ------
All output files will be saved to: /scicore/home/engel0006/GROUP/pool-engel/soft/tomodrgn/tomodrgn/testing/output
Individual command output will be saved to output/commands.output
Individual command errors will be saved to output/commands.error

------  RUNNING  ------
Running command 1/2: 
    tomodrgn train_vae data/10076_both_32_sim.star -o output/01_vae_both_sim --zdim 8 --uninvert-data --seed 42 --log-interval 100 --enc-dim-A 64 --enc-layers-A 2 --out-dim-A 64 --enc-dim-B 32 --enc-layers-B 4 --dec-dim 16 --dec-layers 3 -n 5
    ERROR: see output/commands.error
Running command 2/2: 
    tomodrgn analyze output/01_vae_both_sim 4 --Apix 13.1 
[commands.error.txt](https://github.com/bpowell122/tomodrgn/files/11771464/commands.error.txt)
--ksample 2
    ERROR: see output/commands.error

------  SUMMARY  ------
0/2 completed successfully
2/2 failed with errors
    Failed commands:
    tomodrgn train_vae data/10076_both_32_sim.star -o output/01_vae_both_sim --zdim 8 --uninvert-data --seed 42 --log-interval 100 --enc-dim-A 64 --enc-layers-A 2 --out-dim-A 64 --enc-dim-B 32 --enc-layers-B 4 --dec-dim 16 --dec-layers 3 -n 5
    tomodrgn analyze output/01_vae_both_sim 4 --Apix 13.1 --ksample 2

I paste below the full commands.error file. Is it a problem with the quicktest.py script or am I doing something wrong?

Thank you!

Command 1/2: tomodrgn train_vae data/10076_both_32_sim.star -o output/01_vae_both_sim --zdim 8 --uninvert-data --seed 42 --log-interval 100 --enc-dim-A 64 --enc-layers-A 2 --out-dim-A 64 --enc-dim-B 32 --enc-layers-B 4 --dec-dim 16 --dec-layers 3 -n 5 
Error returncode: 1 
/scicore/home/engel0006/diogori/soft/tomodrgn/tomodrgn_env/lib/python3.11/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/scicore/home/engel0006/diogori/soft/tomodrgn/tomodrgn_env/lib/python3.11/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/scicore/home/engel0006/diogori/soft/tomodrgn/tomodrgn_env/lib/python3.11/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/scicore/home/engel0006/diogori/soft/tomodrgn/tomodrgn_env/lib/python3.11/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
Traceback (most recent call last):
  File "/scicore/home/engel0006/diogori/soft/tomodrgn/tomodrgn_env/bin/tomodrgn", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/scicore/home/engel0006/diogori/soft/tomodrgn/tomodrgn_env/lib/python3.11/site-packages/tomodrgn/__main__.py", line 51, in main
    args.func(args)
  File "/scicore/home/engel0006/diogori/soft/tomodrgn/tomodrgn_env/lib/python3.11/site-packages/tomodrgn/commands/train_vae.py", line 481, in main
    data_generator = DataLoader(data, batch_size=args.batch_size, shuffle=True, num_workers=args.num_workers, prefetch_factor=args.prefetch_factor, persistent_workers=args.persistent_workers, pin_memory=args.pin_memory)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/scicore/home/engel0006/diogori/soft/tomodrgn/tomodrgn_env/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 245, in __init__
    raise ValueError('prefetch_factor option could only be specified in multiprocessing.'
ValueError: prefetch_factor option could only be specified in multiprocessing.let num_workers > 0 to enable multiprocessing, otherwise set prefetch_factor to None.

Command 2/2: tomodrgn analyze output/01_vae_both_sim 4 --Apix 13.1 --ksample 2 
Error returncode: 1 
/scicore/home/engel0006/diogori/soft/tomodrgn/tomodrgn_env/lib/python3.11/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/scicore/home/engel0006/diogori/soft/tomodrgn/tomodrgn_env/lib/python3.11/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/scicore/home/engel0006/diogori/soft/tomodrgn/tomodrgn_env/lib/python3.11/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/scicore/home/engel0006/diogori/soft/tomodrgn/tomodrgn_env/lib/python3.11/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
Traceback (most recent call last):
  File "/scicore/home/engel0006/diogori/soft/tomodrgn/tomodrgn_env/bin/tomodrgn", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/scicore/home/engel0006/diogori/soft/tomodrgn/tomodrgn_env/lib/python3.11/site-packages/tomodrgn/__main__.py", line 51, in main
    args.func(args)
  File "/scicore/home/engel0006/diogori/soft/tomodrgn/tomodrgn_env/lib/python3.11/site-packages/tomodrgn/commands/analyze.py", line 184, in main
    z = utils.load_pkl(zfile)
        ^^^^^^^^^^^^^^^^^^^^^
  File "/scicore/home/engel0006/diogori/soft/tomodrgn/tomodrgn_env/lib/python3.11/site-packages/tomodrgn/utils.py", line 60, in load_pkl
    with open(pkl,'rb') as f:
         ^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/scicore/home/engel0006/GROUP/pool-engel/soft/tomodrgn/tomodrgn/testing/output/01_vae_both_sim/z.4.pkl'
dacolombo commented 1 year ago

Hi @rdrighetto,

I also tried installing TomoDRGN and running the tests in the repo and I had your same issues.

Regarding the first error, I used your same workaround of setting the conda channel to conda-forge.

For the second error, I got your same error with a conda environment having python v3.11 and pytorch v2.0.0. I managed to install a conda enviornment that runs correctly the test scripts provided in the repo by installing python v3.10.4 and pytorch-gpu v1.11.0.

I think the issue is that these commands for installation:

conda create --name tomodrgn "python>=3.7"
...
conda install "pytorch-gpu>=1.11.0" "cudatoolkit>=11.0" -c pytorch

should be more strict with the required versions. Probably setting python=3.10.4 and pytorch-gpu=1.11.0 should solve the issue. I am attaching here the gist with the conda environment specification, so that you could install the software with the following command:

conda env create --file tomodrgn_environment.yml
conda env activate tomodrgn
git clone https://github.com/bpowell122/tomodrgn.git
cd tomodrgn
python -m pip install .

Hope this fixes the issue also for you.

rdrighetto commented 1 year ago

I did a fresh installation using your gist and the tests worked perfectly, thank you @dacolombo!