--replace flag, check if bidsmreye output has been produced already and skip processing of files that already have data [ENH]

cpp-lln-lab / bidsMReye

BIDS app using deepMReye to decode eye motion for fMRI time series data.

https://bidsmreye.readthedocs.io/en/latest/

Other

5 stars 5 forks source link

--replace flag, check if bidsmreye output has been produced already and skip processing of files that already have data [ENH] #168

Closed Michael-Sun closed 2 months ago

Michael-Sun commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues

New feature

Running bidsmreye on a cluster compute gpu node can run into issues if the process gets killed because of memory errors, timeout, or other miscellaneous issues (e.g., administration-level malicious code detection wrongly kills your process). It would be good if bidsmreye can pick up where it left off instead of starting processing all over again.

Unclear documentation

No response

Remi-Gau commented 1 year ago

yeah I was thinking of possibly using a proper workflow manager for this. nipype / pydra or bidsnake?

this may not happen fast but it would be useful to have

Remi-Gau commented 1 year ago

for now I may make the default behavior to reuse pre-existing files and pass a --force flag to overwrite previous results

Remi-Gau commented 1 year ago

Not sure if you are using it. But splitting the processing by running one action at a time may help a bit as well.

https://bidsmreye.readthedocs.io/en/latest/usage_notes.html#named-arguments

Remi-Gau commented 2 months ago

for now I may make the default behavior to reuse pre-existing files and pass a --force flag to overwrite previous results

files that already have an output should now be skipped unless the --force argument is passed in the CLI