Error in rule concat_annotations

[mladen5000]

AFAIK this is one of the remaining steps before the genomes/annotations/kegg_modules.tsv file, which I require in order to do functional analysis much like that done in the tutorial repo.

Removing temporary output genomes/all_contigs.fasta. Using shell: /usr/bin/bash Provided cores: 1 (use --cores to define parallelism) Rules claiming more threads will be scaled down. Provided resources: mem_mb=60000, mem_mib=57221, disk_mb=1000, disk_mib=954, time=2, mem=60, time_min=120 Singularity containers: ignored Select jobs to execute... 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 23/23 [00:00<00:00, 74.69it/s] [Sat Apr 15 09:06:17 2023] Error in rule concat_annotations: jobid: 0 input: genomes/annotations/dram/intermediate_files/MAG18, genomes/annotations/dram/intermediate_files/MAG16, genomes/annotations/dram/intermediate_files/MAG19, genomes/annotations/dram/intermediate_files/MAG21, genomes/annotations/dram/intermediate_files/MAG15, genomes/annotations/dram/intermediate_files/MAG11, genomes/annotations/dram/intermediate_files/MAG04, genomes/annotations/dram/intermediate_files/MAG06, genomes/annotations/dram/intermediate_files/MAG01, genomes/annotations/dram/intermediate_files/MAG17, genomes/annotations/dram/intermediate_files/MAG22, genomes/annotations/dram/intermediate_files/MAG05, genomes/annotations/dram/intermediate_files/MAG13, genomes/annotations/dram/intermediate_files/MAG02, genomes/annotations/dram/intermediate_files/MAG09, genomes/annotations/dram/intermediate_files/MAG14, genomes/annotations/dram/intermediate_files/MAG12, genomes/annotations/dram/intermediate_files/MAG07, genomes/annotations/dram/intermediate_files/MAG20, genomes/annotations/dram/intermediate_files/MAG03, genomes/annotations/dram/intermediate_files/MAG08, genomes/annotations/dram/intermediate_files/MAG10, genomes/annotations/dram/intermediate_files/MAG23 output: genomes/annotations/dram/annotations.tsv, genomes/annotations/dram/rrnas.tsv, genomes/annotations/dram/trnas.tsv

RuleException: FileNotFoundError in file /home/mrasic2/.conda/envs/atlas215/lib/python3.10/site-packages/atlas/workflow/rules/dram.smk, line 120: [Errno 2] No such file or directory: 'genomes/annotations/dram/intermediate_files/MAG18/rrnas.tsv' File "/home/mrasic2/.conda/envs/atlas215/lib/python3.10/site-packages/atlas/workflow/rules/dram.smk", line 120, in rule_concat_annotations File "/home/mrasic2/.conda/envs/atlas215/lib/python3.10/site-packages/atlas/workflow/scripts/utils/io.py", line 204, in pandas_concat File "/home/mrasic2/.conda/envs/atlas215/lib/python3.10/site-packages/atlas/workflow/scripts/utils/io.py", line 131, in _pandas_concat_disck_based File "/home/mrasic2/.conda/envs/atlas215/lib/python3.10/site-packages/pandas/util/_decorators.py", line 311, in wrapper File "/home/mrasic2/.conda/envs/atlas215/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 678, in read_csv File "/home/mrasic2/.conda/envs/atlas215/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 575, in _read File "/home/mrasic2/.conda/envs/atlas215/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 932, in init__ File "/home/mrasic2/.conda/envs/atlas215/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1216, in _make_engine File "/home/mrasic2/.conda/envs/atlas215/lib/python3.10/site-packages/pandas/io/common.py", line 786, in get_handle File "/home/mrasic2/.conda/envs/atlas215/lib/python3.10/concurrent/futures/thread.py", line 58, in run Removing output files of failed job concat_annotations since they might be corrupted: genomes/annotations/dram/annotations.tsv Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message

[SilasK]

Ok, could you check if the annotation for the genome MAG18 run correctly?

check the log logs/dram/run_dram/MAG18.log

Maybe you can delete the whole folder genomes/annotations/dram/intermediate_files/MAG18 and rerun atlas. check if other genomes filed too.

I think it would generate (even with only a header) genomes/annotations/dram/intermediate_files/MAG18/rrnas.tsv

[mladen5000]

I think the header file would occur in a previous version, (either in 2.14.3 or 2.15 or previous DRAM), but since I updated this error started to appear. I checked my previous runs which also have logs that state that

"no rrnas were found so rrnas.tsv not created"

Possibly a change in dram.smk since 2.13? Additionally I know DRAM was upgraded so it could be something on their end that changed.

I was able to manually workaround this by creating rrnas.tsv with just the header so that pandas concat could work. The rule completed successfully and I was able to generate a kegg_modules file.

I am having issues to dram destill still but I will open in a separate issue.

[github-actions[bot]]

There was no activity since some time. I hope your issue is solved in the mean time. This issue will automatically close soon if no further activity occurs.

Thank you for your contributions.