search

prehensilecode / alphafold_singularity

Singularity recipe for AlphaFold
GNU General Public License v3.0
33 stars 12 forks source link

ERROR: No matching distribution found for numpy==1.21.6 #38

Closed johnnytam100 closed 3 weeks ago

johnnytam100 commented 8 months ago

Hi @prehensilecode thank you for making this repo! I met an error that says 

...
Collecting immutabledict==2.0.0 (from -r /app/alphafold/requirements.txt (line 7))
  Downloading immutabledict-2.0.0-py3-none-any.whl.metadata (2.7 kB)
Collecting jax==0.3.25 (from -r /app/alphafold/requirements.txt (line 8))
  Downloading jax-0.3.25.tar.gz (1.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 138.1 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting ml-collections==0.1.0 (from -r /app/alphafold/requirements.txt (line 9))
  Downloading ml_collections-0.1.0-py3-none-any.whl.metadata (18 kB)
ERROR: Ignored the following versions that require a different python version: 1.21.2 Requires-Python >=3.7,<3.11; 1.21.3 Requires-Python >=3.7,<3.11; 1.21.4 Requires-Python >=3.7,<3.11; 1.21.5 Requires-Python >=3.7,<3.11; 1.21.6 Requires-Python >=3.7,<3.11
ERROR: Could not find a version that satisfies the requirement numpy==1.21.6 (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0, 1.11.1, 1.11.2, 1.11.3, 1.12.0, 1.12.1, 1.13.0, 1.13.1, 1.13.3, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.16.5, 1.16.6, 1.17.0, 1.17.1, 1.17.2, 1.17.3, 1.17.4, 1.17.5, 1.18.0, 1.18.1, 1.18.2, 1.18.3, 1.18.4, 1.18.5, 1.19.0, 1.19.1, 1.19.2, 1.19.3, 1.19.4, 1.19.5, 1.20.0, 1.20.1, 1.20.2, 1.20.3, 1.21.0, 1.21.1, 1.22.0, 1.22.1, 1.22.2, 1.22.3, 1.22.4, 1.23.0rc1, 1.23.0rc2, 1.23.0rc3, 1.23.0, 1.23.1, 1.23.2, 1.23.3, 1.23.4, 1.23.5, 1.24.0rc1, 1.24.0rc2, 1.24.0, 1.24.1, 1.24.2, 1.24.3, 1.24.4, 1.25.0rc1, 1.25.0, 1.25.1, 1.25.2, 1.26.0b1, 1.26.0rc1, 1.26.0, 1.26.1, 1.26.2, 1.26.3, 1.26.4, 2.0.0b1)
ERROR: No matching distribution found for numpy==1.21.6
+ cd /opt/conda/lib/python3.8/site-packages
/.post.script: 75: cd: can't cd to /opt/conda/lib/python3.8/site-packages
FATAL:   While performing build: while running engine: exit status 2

Do you know how to modify the definition file to make it works?

JuergenUniVie commented 8 months ago

I have the same problem. tried to use numpy 1.22.0 (requirements.txt) But then completely different errors occur...

Collecting numpy==1.22.0 (from -r /app/alphafold/requirements.txt (line 10)) Downloading numpy-1.22.0.zip (11.3 MB) 11 .3/11.3 MB 114.9 MB/s eta 0 :00:00 Installing build dependencies...done Getting requirements to build wheel ... error error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> [33 lines of output] Traceback (most recent call last): File "/opt/conda/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in Main() File "/opt/conda/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 112, in get_requires_for_build_wheel backend = _build_backend() ^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend obj = import_module(mod_path) ^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.12/importlib/init.py", line 90, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^ File "", line 1387, in _gcd_import File "", line 1360, in _find_and_load File "", line 1310, in _find_and_load_unlocked File "", line 488, in _call_with_frames_removed File "", line 1387, in _gcd_import File "", line 1360, in _find_and_load File "", line 1331, in _find_and_load_unlocked File "", line 935, in _load_unlocked File "", line 994, in exec_module File "", line 488, in _call_with_frames_removed File "/tmp/pip-build-env-zk0r3axd/overlay/lib/python3.12/site-packages/setuptools/init.py", line 16, in import setuptools.version File "/tmp/pip-build-env-zk0r3axd/overlay/lib/python3.12/site-packages/setuptools/version.py", line 1, in import pkg_resources File "/tmp/pip-build-env-zk0r3axd/overlay/lib/python3.12/site-packages/pkg_resources/init.py", line 2172, in register_finder(pkgutil.ImpImporter, find_on_path) ^^^^^^^^^^^^^^^^^^^ AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'? [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

JuergenUniVie commented 8 months ago

Adjusting the Singularity.def and changing the MiniConda version to my Python version solved the problem. Use my Python version instead of latest.

Install Miniconda package manager.

wget -q -P /tmp \ https://repo.anaconda.com/miniconda/Miniconda3-py38_23.11.0-2-Linux-x86_64.sh\ && bash /tmp/Miniconda3-py38_23.11.0-2-Linux-x86_64.sh -b -p /opt/conda \ && rm /tmp/Miniconda3-py38_23.11.0-2-Linux-x86_64.sh

-> https://repo.anaconda.com/miniconda/

johnnytam100 commented 8 months ago

Hi @JuergenUniVie ! Thank you for the comment! I have changed inside Singularity.def to my python version but still didn't work..

For the miniconda package, do you mean to install it?

JuergenUniVie commented 8 months ago

Hi @johnnytam100

I changed the lines with MiniConda in Singularity.def so that the Conda version uses Python3.8 and not the latest version. I'm using Python 3.8.

python --version Python 3.8.17

Change the lines so that the appropriate MiniConda version matches your installed Python version:

Singularity.dev:

Install Miniconda package manager.

wget -q -P /tmp \ https://repo.anaconda.com/miniconda/Miniconda3-py38_23.11.0-2-Linux-x86_64.sh \ && bash /tmp/Miniconda3-py38_23.11.0-2-Linux-x86_64.sh -b -p /opt/conda \ && rm /tmp/Miniconda3-py38_23.11.0-2-Linux-x86_64.sh

if you use Python 3.10 then replace with:

https://repo.anaconda.com/miniconda/Miniconda3-py310_23.11.0-2-Linux-x86_64.sh && bash/tmp/Miniconda3-py310_23.11.0-2-Linux-x86_64.sh .....

JuergenUniVie commented 8 months ago

However, it may only work with Python 3.8 as I find several references to it, which you would then have to adapt to your Python version.

pdbfixer \ pip \ python=3.8 \ && conda clean --all --force-pkgs-dirs --yes

/bin/cp -r . /app/alphafold

wget -q -P /app/alphafold/alphafold/common/ \ https://git.scicore.unibas.ch/schwede/openstructure/-/raw/7102c63615b64735c4941278d92b554ec94415f8/modules/mol/alg/src/stereo_chemical_props.txt

Install pip packages.

N.B. The URL specifies the list of jaxlib releases.

pip3 install --upgrade pip --no-cache-dir \ && pip3 install -r /app/alphafold/requirements.txt --no-cache-dir \ && pip3 install --upgrade --no-cache-dir \ jax==0.3.25 \ jaxlib==0.3.25+cuda11.cudnn805 \ -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html

Apply OpenMM patch.

cd /opt/conda/lib/python3.8/site-packages patch -p0 < /app/alphafold/docker/openmm.patch

johnnytam100 commented 7 months ago

Thanks! It worked! My final Singularity.def looks like this:

Bootstrap: docker
From: nvidia/cuda:11.1.1-cudnn8-runtime-ubuntu18.04
Stage: spython-base

%files
. /app/alphafold
%post
# Copyright 2021 DeepMind Technologies Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# FROM directive resets ARGS, so we specify again (the value is retained if
# previously set).

apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y \
build-essential \
cmake \
cuda-command-line-tools-11-1 \
git \
hmmer \
kalign \
tzdata \
wget \
&& rm -rf /var/lib/apt/lists/* \
&& apt-get autoremove -y \
&& apt-get clean

# Compile HHsuite from source.
/bin/rm -rf /tmp/hh-suite \
&& git clone --branch v3.3.0 https://github.com/soedinglab/hh-suite.git /tmp/hh-suite \
&& mkdir /tmp/hh-suite/build \
&& cd /tmp/hh-suite/build \
&& cmake -DCMAKE_INSTALL_PREFIX=/opt/hhsuite .. \
&& make -j 4 && make install \
&& ln -s /opt/hhsuite/bin/* /usr/bin \
&& cd / \
&& /bin/rm -rf /tmp/hh-suite

# Install Miniconda package manager.
wget -q -P /tmp \
https://repo.anaconda.com/miniconda/Miniconda3-py38_23.11.0-2-Linux-x86_64.sh \
&& bash /tmp/Miniconda3-py38_23.11.0-2-Linux-x86_64.sh -b -p /opt/conda \
&& rm /tmp/Miniconda3-py38_23.11.0-2-Linux-x86_64.sh

# Install conda packages.
PATH="/opt/conda/bin:/usr/local/cuda-11.1/bin:$PATH"
conda install -qy conda==4.13.0 \
&& conda install -y -c conda-forge \
openmm=7.5.1 \
cudatoolkit==11.1.1 \
pdbfixer \
pip \
python=3.8 \
&& conda clean --all --force-pkgs-dirs --yes

### /bin/cp -r . /app/alphafold

wget -q -P /app/alphafold/alphafold/common/ \
https://git.scicore.unibas.ch/schwede/openstructure/-/raw/7102c63615b64735c4941278d92b554ec94415f8/modules/mol/alg/src/stereo_chemical_props.txt

# Install pip packages.
# N.B. The URL specifies the list of jaxlib releases.
pip3 install --upgrade pip  --no-cache-dir \
&& pip3 install -r /app/alphafold/requirements.txt --no-cache-dir \
&& pip3 install --upgrade --no-cache-dir \
jax==0.3.25 \
jaxlib==0.3.25+cuda11.cudnn805 \
-f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html

# Apply OpenMM patch.
cd /opt/conda/lib/python3.8/site-packages
patch -p0 < /app/alphafold/docker/openmm.patch

# Add SETUID bit to the ldconfig binary so that non-root users can run it.
chmod u+s /sbin/ldconfig.real

### SETUID bit does not matter: Apptainer does not allow suid commands
### Workaround below is to use /mnt/out/ld.so.cache for the ld cache file

%environment
export PATH="/opt/conda/bin:/usr/local/cuda-11.1/bin:$PATH"
%runscript
cd /app/alphafold
ldconfig -C /mnt/output/ld.so.cache
exec python /app/alphafold/run_alphafold.py "$@"
# %startscript
# cd /app/alphafold
# exec python /app/alphafold/run_alphafold.py "$@"