leylabmpi / Struo2

Scalable creating/updating of metagenome profiling databases
MIT License
57 stars 8 forks source link

Error updating Kraken2 GTDB Database with MAGs - MissingOutputException in rule kraken2_cp_to_tmp in file #42

Closed PeterCx closed 1 year ago

PeterCx commented 1 year ago

Hi Nick,

I am encountering a problem when using snakemake to update my Kraken2 database. See the error bwlo.

Waiting at most 5 seconds for missing files. MissingOutputException in rule kraken2_cp_to_tmp in file /workspace/pot/peterc/Equine/Struo2/bin/ Job 2 completed successfully, but some output files are missing. Missing files after 5 seconds. is the case, consider to increase the wait time with --latency-wait: tmp/db_update_tmp/peterc/Struo2_112566273/db_update/kraken2/taxonomy/nodes.dmp tmp/db_update_tmp/peterc/Struo2_112566273/db_update/kraken2/taxonomy/names.dmp Removing output files of failed job kraken2_cp_to_tmp since they might be corrupted: tmp/db_update_tmp/peterc/Struo2_112566273/db_update/kraken2/library, tmp/db_update_tmp/peterc/Str Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message

Pipeline failed! Creating report... --- I/O info --- tmp_dir: tmp/db_update_tmp/peterc/Struo2_112566273/ % used size: 82.1 % used inodes: 0.0 output_dir: /workspace/pot/peterc/Equine/Struo2/Output/ % used size: 82.1 % used inodes: 0.0 --- Summarizing job logs --- 2023-01-27 14:10:17,527 - Parsing log: /workspace/pot/peterc/Equine/Struo2/.snakemake/log/2023-01-27T133338.647415.snakemake.log 2023-01-27 14:10:17,873 - Cannot find qacct log: /var/lib/gridengine/default/common/accounting; Skipping --- No email provided; not sending a notification --- Complete log: .snakemake/log/2023-01-27T133338.647415.snakemake.log

I cant detect what the exact problem is. See attached the files associated with this. I am using the following command to run the pipeline:

snakemake --use-conda --cores 20 --configfile config-update.yaml

job_report.csv config-update.txt Sample_Table.txt

Your help is greatly appreciated.

Kind regards,

P

PeterCx commented 1 year ago

There was an error in the config file.

Saved databases that will be updated

kraken2_db: library: /workspace/pot/peterc/Equine/Struo2/custom_dbs taxonomy: /workspace/pot/peterc/Equine/Struo2/custom_dbs

Should be:

kraken2_db: library: /workspace/pot/peterc/Equine/Struo2/custom_dbs/library taxonomy: /workspace/pot/peterc/Equine/Struo2/custom_dbs/taxonomy

Many thanks