search

metagenome-atlas / atlas

ATLAS - Three commands to start analyzing your metagenome data
https://metagenome-atlas.github.io/
BSD 3-Clause "New" or "Revised" License
364 stars 97 forks source link

Setting eggNOG_use_virtual_disk to true #701

Closed Roxy-mzh closed 10 months ago

Roxy-mzh commented 10 months ago

Is your feature request related to a problem? Please describe.

When trying to run the genecatalog for on another dataset, after setting the eggNOG_use_virtual_disk to true, it tries to re-download the eggnog.db and since the database is already downloaded, ATLAS run gets an error of cp: cannot create regular file 'XXX/eggnog_proteins.dmnd': File exists. Therefore, it seems like every time, this database should be deleted before a re-run.

Describe the solution you'd like

It would be great to have the option of skipping this delete/ re-download for each run.

Additional context Add any other context or screenshots about the feature request here.

SilasK commented 10 months ago

In which rule does the copy error appear? When using the virtual disk. the database is copied to the virtual disk so the copy error might also be there.

The disk is by default virtual_disk: "/dev/shm"

Does this exist on your system?

It is true that it is copied for each job. Do you have the same shared memory for all jobs?

Roxy-mzh commented 10 months ago

Error in rule eggnog_annotation.

Yes, we have this /dev/shm on our system and the memory is shared for all jobs.

SilasK commented 10 months ago

Fixed in the masterbranch. could you try? I guess there could be the case that one job starts to copy the database and others want access it. but it becomes complicated to solve this for all cases.

Roxy-mzh commented 9 months ago

Sorry for the late reply. Thank you it works now!