katzlabbrandeis / blech_clust

GNU General Public License v3.0
7 stars 4 forks source link

Make FAQ with deleting results.log for blech_run_process.py #171

Closed emmalala123 closed 2 months ago

emmalala123 commented 5 months ago

The bash command sometimes gives up half way through :(

Mraymon5 commented 3 months ago

I think I was running into this too; if I ran blech_run_process.sh, or blech_process_parallel.sh, with a results.log file pre-existing, then Bash just printed "0" and shut down.

Deleting the results.log file immediately fixed the problem with running blech_process_parallel.sh, and allows blech_run_process.sh to run one "retry", but on subsequent retries it just starts printing "0" again and stopping.

Modifying blech_process_parallel.sh so that it made a unique name for results.log actually worked for both scripts: it allows blech_run_process.sh to create a new result.log for each retry cycle, which stops the erroneous behavior.

Because blech_process_parallel.sh is generated during the running of blech_clust, the issue presumably needs to be addressed at the root, rather than trying to fix it in the data folder/temp every time.

abuzarmahmood commented 3 months ago

Sorry @emmalala123, I missed this issue. I think it has to do with what parallel considers a failed run. I'm expecting that parallel misunderstands failed processing of a channel and marks it as successful in the log. So that when the process is re-run, it simply goes through the log and says everything worked. We might either have to deal more explicitly with exit codes (only give a successful code at the end of blech_process.py), or figure out how to make our own log.

abuzarmahmood commented 3 months ago

@Mraymon5 I think creating a new results log for every run will force it to rerun all channels...is that what you see?

Mraymon5 commented 3 months ago

Current master does not deal with results.log already being present. Add code to delete results.log if already present or ask user when running blech_process if they want to use current results.log