search

metagenome-atlas / atlas

ATLAS - Three commands to start analyzing your metagenome data
https://metagenome-atlas.github.io/
BSD 3-Clause "New" or "Revised" License
376 stars 98 forks source link

Atlas was not able to download a key database #738

Closed ShevchenkoAlla closed 1 week ago

ShevchenkoAlla commented 2 weeks ago

Hello, I have tried to run atlas on the server a few times but every time I have the same problem, it seems to be unable to download a database. Are there any solutions? So I started with "conda activate atlas-dev", then "atlas init -w /dir --db-dir /dir", and then "atlas run all --working-dir /dir --config-file /dir/config.yaml --jobs 10".

So every time it left me with this:

log: logs/dram/download_dram.log (check log file(s) for error details) conda-env: /dir/databases/condaenvs/c9d07354e296f0f3e7ea7066304ef57b shell: DRAM-setup.py prepare_databases --output_dir /dir/databases/DRAM/db --threads 8 --verbose --skip_uniref &> logs/dram/download_dram.log ; DRAM-setup.py export_config --output_file /dir/databases/DRAM/DRAM.config (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!) Logfile logs/dram/download_dram.log:

/dir/databases/condaenvs/c9d07354e296f0f3e7ea7066304ef57b/lib/python3.11/site-packages/mag_annotator/database_handler.py:169: UserWarning: Database does not exist at path None warnings.warn( /dir/databases/condaenvs/c9d07354e296f0f3e7ea7066304ef57b/lib/python3.11/site-packages/mag_annotator/database_handler.py:126: UserWarning: Database does not exist at path None warnings.warn("Database does not exist at path %s" % description_loc) 2024-10-09 02:26:11,995 - Starting the process of downloading data 2024-10-09 02:26:11,995 - Skipping UniRef 2024-10-09 02:26:11,995 - The kegg_loc argument was not used to specify a downloaded kegg file, and dram can not download it its self. So it is assumed that the user wants to set up DRAM without it 2024-10-09 02:26:11,995 - The gene_ko_link_loc argument was not used to specify a downloaded gene_ko_link file, and dram can not download it its self. So it is assumed that the user wants to set up DRAM without it 2024-10-09 02:26:11,995 - Database preparation started 2024-10-09 02:26:11,995 - Downloading kofam_hmm 2024-10-09 02:31:10,876 - Downloading kofam_ko_list 2024-10-09 02:31:44,463 - Downloading pfam 2024-10-09 02:40:28,769 - Downloading pfam_hmm 2024-10-09 02:40:29,582 - Downloading dbcan 2024-10-09 02:42:39,339 - Something went wrong with the download of the url: http://bcb.unl.edu/dbCAN2/download/dbCAN-HMMdb-V11.txt 2024-10-09 02:42:39,339 - <urlopen error [Errno 110] Connection timed out> downloading ftp://ftp.genome.jp/pub/db/kofam/profiles.tar.gz downloading ftp://ftp.genome.jp/pub/db/kofam/ko_list.gz downloading ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.full.gz downloading ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.hmm.dat.gz downloading http://bcb.unl.edu/dbCAN2/download/dbCAN-HMMdb-V11.txt Traceback (most recent call last): File "/dir/databases/condaenvs/c9d07354e296f0f3e7ea7066304ef57b/bin/DRAM-setup.py", line 187, in args.func(**args_dict) File "/dir/databases/condaenvs/c9d07354e296f0f3e7ea7066304ef57b/lib/python3.11/site-packages/mag_annotator/database_processing.py", line 592, in prepare_databases locs[i] = download_functions[i]( ^^^^^^^^^^^^^^^^^^^^^^ File "/dir/databases/condaenvs/c9d07354e296f0f3e7ea7066304ef57b/lib/python3.11/site-packages/mag_annotator/database_processing.py", line 169, in download_dbcan download_file(link_path, dbcan_hmm, logger, verbose=verbose) File "/dir/databases/condaenvs/c9d07354e296f0f3e7ea7066304ef57b/lib/python3.11/site-packages/mag_annotator/utils.py", line 33, in download_file raise URLError("DRAM whas not able to download a key database, check the logg for details") urllib.error.URLError: <urlopen error DRAM whas not able to download a key database, check the logg for details>

Removing output files of failed job dram_download since they might be corrupted: /dir/databases/DRAM/db Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Note the path to the log file for debugging. Documentation is available at: https://metagenome-atlas.readthedocs.io Issues can be raised at: https://github.com/metagenome-atlas/atlas/issues Complete log: .snakemake/log/2024-10-08T142448.194252.snakemake.log [Atlas] CRITICAL: Command 'snakemake --snakefile /dir/atlas/atlas/workflow/Snakefile --directory /dir --rerun-triggers mtime --jobs 10 --rerun-incomplete --configfile '/dir/config.yaml' --nolock --show-failed-logs --use-conda --conda-prefix /dir/databases/conda_envs --resources mem=1903 mem_mb=1949357 java_mem=1618 --scheduler greedy all ' returned non-zero exit status 1.

atlas, version 2.18.2+17.g22d78b98

I can't find any log files either, it says there is no such directory. Which is strange.

Thank you, Best

SilasK commented 1 week ago

Hey, we had problems with dram download before. In the meantime they should be solved. But the version 2.19 was not on bioconda. It should be in an hour or so.

Can you simply upodate atlas. mamba install metagenome-atlas=2.19

ShevchenkoAlla commented 1 week ago

Hey, we had problems with dram download before. In the meantime they should be solved. But the version 2.19 was not on bioconda. It should be in an hour or so.

Can you simply upodate atlas. mamba install metagenome-atlas=2.19

Hi, I can't update it

conda install metagenome-atlas=2.19 /home/m/miniconda3/lib/python3.12/site-packages/conda/base/context.py:198: FutureWarning: Adding 'defaults' to channel list implicitly is deprecated and will be removed in 25.3.

To remove this warning, please choose a default channel explicitly with conda's regular configuration system, e.g. by adding 'defaults' to the list of channels:

conda config --add channels defaults

For more information see https://docs.conda.io/projects/conda/en/stable/user-guide/configuration/use-condarc.html

deprecated.topic( /home/m/miniconda3/lib/python3.12/site-packages/conda/base/context.py:198: FutureWarning: Adding 'defaults' to channel list implicitly is deprecated and will be removed in 25.3.

To remove this warning, please choose a default channel explicitly with conda's regular configuration system, e.g. by adding 'defaults' to the list of channels:

conda config --add channels defaults

For more information see https://docs.conda.io/projects/conda/en/stable/user-guide/configuration/use-condarc.html

deprecated.topic( Channels:

PackagesNotFoundError: The following packages are not available from current channels:

Current channels:

To search for alternate channels that may provide the conda package you're looking for, navigate to

https://anaconda.org

and use the search bar at the top of the page.