search

gitter-lab / SINGE

Gene regulatory network reconstruction from pseudotemporal single-cell gene expression data
MIT License
11 stars 6 forks source link

Delete Temporary mat files at the end of run_SINGE_GLG_test.sh #61

Open atuldeshpande opened 3 years ago

atuldeshpande commented 3 years ago

Issue: Depending on the data size, large TempMat.mat files are created during SINGE_GLG_Test, which could overrun user's storage budget leading to subsequent jobs being held. Fix: We should include the line `eval "rm Temp.mat"` near the end of run_SINGE_GLG_Test.sh

EDIT: The rm should be more targeted, especially if the storage is shared, with only Temp_<ID>.mat to be deleted.

agitter commented 3 years ago

The end of currently has SINGE_GLG_Test.m https://github.com/gitter-lab/SINGE/blob/9c59a1e7d442bccf4a3fab8b0eaa06c920a3e857/code/SINGE_GLG_Test.m#L92

Is that line not working?

atuldeshpande commented 3 years ago

I think there's a case to be made to clean up if for any reason a job fails, not reaching this point in MATLAB.

agitter commented 3 years ago

I agree that we should still delete temporary files even if a job fails. However, is it possible to do that in a way that also cleans up the temporary files if the job fails when run through MATLAB instead of the shell script? For example, can we catch errors in this function and delete the temporary file if there is an error?

agitter commented 3 years ago

Do we also need to delete the file TempMat.mat that is created here? https://github.com/gitter-lab/SINGE/blob/9c59a1e7d442bccf4a3fab8b0eaa06c920a3e857/code/parseParams.m#L76-L86