FileExistsError: [Errno 17] File exists: './dramv-annotate' when specifying output dir

[flannsmith]

An error is being thrown when I specify the output dir when the folder exists.

DRAM-v.py annotate -i /backup/user_data/linda/f_dynamics/vs2-pass2/final-viral-combined-for-dramv.fa -v /backup/user_data/linda/f_dynamics/vs2-pass2/for-dramv/viral-affi-contigs-for-dramv.tab -o ./dramv-annotate --skip_trnascan --threads 10 --min_contig_size 1000

  File "/home/linda/anaconda3/envs/DRAM/lib/python3.9/site-packages/mag_annotator/annotate_vgfs.py", line 384, in annotate_vgfs
    mkdir(output_dir)
FileExistsError: [Errno 17] File exists: './dramv-annotate'

A simple remedy in annotate_vgfs.py on line 384 would be to check if the folder path exists

import os

if not os.path.exists(output_dir):
        mkdir(output_dir)
    else:
        contig_dir = path.join(output_dir, 'vMAGs')

[shafferm]

We decided to throw this error whenever a user tries to use a folder that already exists. There are a few risks to using a directory that already exists such as writing over user files that already exist with the same name as what DRAM would want to name files so we chose to let it throw an error here. That way we never endanger files users have already made. I have it on my to do list to add a flag that would all the user to chose to overwrite a folder but this is not a high priority right now.

[rdenise]

Hello everyone, I just wanted to go back on this problem because with DRAM-v.py distill, there is also this error and it occurs because I use it in a snakemake workflow and so snakemake create the folder of the output file before running the rule. So snakemake create the empty folder and then DRAM try to create the folder and then Error. But the folder is empty because it was just create 1 second ago by snakemake. That means that dram could not be used directly in any snakemake workflow directly.

[rmFlynn]

This is a really annoying problem that I have need to deal with often. There are workarounds and if snakemake is up-to-date this should happen less often. I have not used distill as part of snake make for the last month, but I will be digging it up and messing with it soon to get it nailed down when this happens. Like if you specify the distillate directory as an output directory, snakemake should not make the directory before dram. Note also that there are some old versions of snakemake out there, and they are different. The solution could be as simple as above, or a bit more complex. I will need to find the exact context to decide.

[rmFlynn]

Time heals all bugs, and snake make has fixed this one for good. As far as I am aware.

[Sidduppal]

Not sure if it's completely resolved. I'm using the latest version of snakemake (7.20.0) and still facing the FileExistsError

[rmFlynn]

I was not aware that this is the case, it may take me a moment to get to it but I will make absolutely sure that the latest version of snake make is working with DRAM or provide a suitable workaround

[rmFlynn]

@dmitrisvetlov Don't worry about this immediately DRAM is not SnakeMake friendly yet and while it is a goal for snakemake to be a first class citizen it is unfortunately a dram 2 things. Because we don't officially support snakemake this is an enhancement. That said @Sidduppal we will get to this within a week or two.

[Sidduppal]

No worries. Currently, I'm using a simple hack of removing the directories in the snakefile before executing DRAM. A quick fix for you guys could be to include a --force flag that can overwrite the directories. I went through the DRAM codebase and it seemed like this was in the works but never implemented (link).