search

1edv / evolution

This repository contains the code for our manuscript - 'The evolution, evolvability, and engineering gene regulatory DNA'
MIT License
93 stars 27 forks source link

Problem installing environment from .yml file #2

Closed lisadratva closed 3 years ago

lisadratva commented 3 years ago

Hi 1edv,

Thanks for solving the Docker issue so quickly! I've run into another problem when trying to create the environment from your evolution_env.yml. The process seems stuck importing the libraries, with the following message being continuously displayed:

...
Warning: you have pip-installed dependencies in your environment file, but you do not list pip itself as one of your conda dependencies.  Conda may not use the correct pip to install your packages, and they may end up in the wrong place.  Please add an explicit pip dependency.  I'm adding one for you, but still nagging you.
Warning: you have pip-installed dependencies in your environment file, but you do not list pip itself as one of your conda dependencies.  Conda may not use the correct pip to install your packages, and they may end up in the wrong place.  Please add an explicit pip dependency.  I'm adding one for you, but still nagging you.
...

I've aborted the terminal after a few hours of this. This happened both in a Windows and in a Linux environment on different machines.

If I add pip as a dependency to the yml file, I get Solving environment: failed, ResolvePackageNotFound errors for a number of packages. I've also tried adding the channels to my environment in the order specified by your yml file, then installing via conda or pip, and having the requirements.txt installed beforehand, but this didn't solve it either.

Thank you for your support!

1edv commented 3 years ago

Hi lisadratva,

If creating the conda environment from the .yml is really slow for you, then you have (at least) two other options for running the code in the repository manually :

    1. Create a new empty conda environment.
    2. Activate the new empty conda environments.
    3. pip install -r requirements.txt within the environment. (The requirements.txt file is in the app folder in the repository)
    4. This new conda environment should be equivalent to the one from the .yml file and should have most (if not all) of the dependencies installed for running any code in the repository.
  2. Run the notebooks from within the docker container you used for loading the model.
Hopefully, one of these two options should resolve the issue and allow you to run the block of code you are interested in. If not, please feel free to re-open the issue. Good luck !
lisadratva commented 3 years ago

Hi edv1,

Thank you for your answer. Regarding your suggestion 1), I still run into installation problems. Could you please share which python, pip, and jupyter versions need to be installed for the empty conda environment? The python default is the most recent (e.g. python 3.9+), but this clashes with tensorflow, which supports only python <=3.8. When trying with a lower python version, I still have unresolved package errors when running the notebook:

        ResolvePackageNotFound:
          - openssl==1.1.1=h7b6447c_0
          - qt==4.8.7=2
          - rpy2==3.1.0=py36r36hc1659b7_0

I have also noticed that files named aux are problematic on Windows (see this issue for an explanation) - this means certain files aren't created when cloning the repo (see output below). I was able to circumvent this by renaming the file aux.py to aux1.py and copying it manually from the repo.

Cloning into 'evolution'...
remote: Enumerating objects: 759, done.
remote: Counting objects: 100% (342/342), done.
remote: Compressing objects: 100% (218/218), done.

Receiving objects: 100% (759/759), 26.92 MiB | 920.00 KiB/s, done.
Resolving deltas: 100% (455/455), done.
error: unable to create file manuscript_code/__pycache__/aux.cpython-36.pyc: No such file or directory
error: unable to create file manuscript_code/aux.py: No such file or directory
error: unable to create file manuscript_code/aux.pyc: No such file or directory
fatal: unable to checkout working tree
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'

Thanks for your answer regarding the package versions and have a nice day!

1edv commented 3 years ago

Hi lisadratva,

Hi edv1,

Thank you for your answer. Regarding your suggestion 1), I still run into installation problems. Could you please share which python, pip, and jupyter versions need to be installed for the empty conda environment? The python default is the most recent (e.g. python 3.9+), but this clashes with tensorflow, which supports only python <=3.8. When trying with a lower python version, I still have unresolved package errors when running the notebook:

      ResolvePackageNotFound:
        - openssl==1.1.1=h7b6447c_0
        - qt==4.8.7=2
        - rpy2==3.1.0=py36r36hc1659b7_0

python 3.6.7 pip 19.3 jupyter 4.4.0 Yes, you are right : rpy2 was used in the figure_orthologs.ipynb notebook to compute p-values in a cell (and the web-app did not require it so it looks like it was missed from the requirements.txt). If you are running that R cell in the notebook, manually installing rpy2 should work; if not, those three imports can be removed without affecting the rest of the code.

I have also noticed that files named aux are problematic on Windows (see this issue for an explanation) - this means certain files aren't created when cloning the repo (see output below). I was able to circumvent this by renaming the file aux.py to aux1.py and copying it manually from the repo.

Cloning into 'evolution'...
remote: Enumerating objects: 759, done.
remote: Counting objects: 100% (342/342), done.
remote: Compressing objects: 100% (218/218), done.

Receiving objects: 100% (759/759), 26.92 MiB | 920.00 KiB/s, done.
Resolving deltas: 100% (455/455), done.
error: unable to create file manuscript_code/__pycache__/aux.cpython-36.pyc: No such file or directory
error: unable to create file manuscript_code/aux.py: No such file or directory
error: unable to create file manuscript_code/aux.pyc: No such file or directory
fatal: unable to checkout working tree
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'

Thanks for your answer regarding the package versions and have a nice day!

Thanks so much - this is very helpful, I will be careful and avoid the aux name from now on !