search

SystemsGenetics / GEMmaker

A workflow for construction of Gene Expression count Matrices (GEMs). Useful for Differential Gene Expression (DGE) analysis and Gene Co-Expression Network (GCN) construction
https://gemmaker.readthedocs.io/en/latest/
MIT License
33 stars 16 forks source link

Fail on resume due to file not being present in stage directory #268

Open JohnHadish opened 1 year ago

JohnHadish commented 1 year ago

Description of the bug

GEMmaker stopped due to an unrelated issue with a file format. After fixing that issue, I restarted GEMmaker. GEMmaker then gave this error:

No such file: /scidas/unmigrated/John/18-Random_Forest_Arabidopsis_Attributes/04-GEMmaker/03-KAMIAK/work/GEMmaker/stage/SRX7267319.sample.csv

which is an issue with the Staging File. I located the file, whcih was in the process directory and moved it to the stage. This did not solve the issue. Apon restart GEMmaker returned the file to the process file and threw the same error.

The file in the incorrect directory was not related to the initial error, and I suspect that GEMmaker was just stopped at an unfortunate time where it thinks the file should be in process but actually needs it in stage.

Banaided my GEMmaker run by telling GEMmaker to ignore this file.

Note: Not using most recent version of GEMmaker, but when looking at difference between versions nothing related to

Command used and terminal output

No response

Relevant files

No response

System information

No response

spficklin commented 1 year ago

I believe on restart, any files in the process directory get moved back to the stage automatically. I could be wrong but I remember at some point GEMmaker did that. I think we need a little script that will clean up samples that are weirded out. Even despite all of our debugging to make GEMmaker as robust as can be there are always weird fringe cases that throw it off. A simple cleanup script to remove directories for a given sample should allow GEMmaker to restart the samples that are weireded out