metagenome-atlas / atlas

ATLAS - Three commands to start analyzing your metagenome data
https://metagenome-atlas.github.io/
BSD 3-Clause "New" or "Revised" License
364 stars 97 forks source link

MissingOutputException #710

Closed topfm closed 8 months ago

topfm commented 8 months ago
rule run_decontamination:
    input: MG1052/sequence_quality_control/MG1052_filtered_se.fastq.gz, ref/genome/1/summary.txt
    output: MG1052/sequence_quality_control/MG1052_clean_se.fastq.gz, MG1052/sequence_quality_control/contaminants/PhiX_se.fastq.gz, MG1052/sequence_quality_control/MG1052_decontamination_reference_stats.txt
    log: MG1052/logs/QC/decontamination.err, MG1052/logs/QC/decontamination.log
    jobid: 4
    benchmark: logs/benchmarks/QC/decontamination/MG1052.txt
    reason: Missing output files: MG1052/sequence_quality_control/MG1052_clean_se.fastq.gz
    wildcards: sample=MG1052
    threads: 8
    resources: tmpdir=/tmp, mem=60, java_mem=51, mem_mb=60000, mem_mib=57221, time_min=300, runtime=300

Activating conda environment: databases/conda_envs/a421f6894bac398293a8d80fc3698e8d_

Waiting at most 120 seconds for missing files.
MissingOutputException in rule run_decontamination in file /home/mtopf/temp-mtopf/2023-11-07-atlas/github_atlas/atlas/atlas/workflow/rules/qc.smk, line 300:
Job 4  completed successfully, but some output files are missing. 

Missing files after 120 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:
MG1052/sequence_quality_control/contaminants/PhiX_se.fastq.gz
Removing output files of failed job run_decontamination since they might be corrupted:
MG1052/sequence_quality_control/MG1052_clean_se.fastq.gz, MG1052/sequence_quality_control/MG1052_decontamination_reference_stats.txt
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

Atlas version 2.18.1+2.gce224044

other relevant information I am using ATLAS installed by github per readthedocs. I am testing ATLAS on one metagenomic sample, MG1052.fastq. Interestingly, ATLAS qc and binning, but not 'genomes' worked when I used an older version of atlas, 2.14.

SilasK commented 8 months ago

I should make the missing file optional. As not all files might have contaminants as ot is your case apparently.

Let me change it in the code.

SilasK commented 8 months ago

It should be fixed in the main branch. if you installed atlas with pip --editabl you can simply pull the changes with git pull.