[x] For the epoch pipeline (before merging with iEEGPrep), change the tmp dir configuration. For this, create two variables in the config file: 'local_scratch' and 'local_scratch_env'. The first one is a path a the latest a system env variable. Then in the snakemake file, try to read the env option first, if fails, read the path, else: don't copy the file (probably set to None). Therefore, make the file copying optional.
[x] For the epoch pipeline (before merging with iEEGPrep). There's the chance that storage gets full if multiple patients are copying the edf file to tmp dir. To solve this potential issue, create a new resources variable (something like 'tmpdir_size') under 'Resources' for the epoch rule. Then, the user SHOULD indicate the total amount of bytes available through the CLI using --resources (Command line interface — Snakemake 7.25.0 documentation). This will make snakemake to parallelize as many subjects or iterations as possible without saturating the memory.
[x] Put all the preprocessing steps + the epoch rule into one group.
[x] Run benchmark to figure out the resources taken by each rule and make the parallelization more efficient. Extra: You could also use group components from the CLI (Command line interface — Snakemake 7.25.0 documentation), which just, instead of submitting one job per subject), can submit one job per each n subjects (it's on you to figure out the n number). However, this is something on the CLI so it would be just for me, possibly add a note about it in the documentation.
[x] For the epoch pipeline (before merging with iEEGPrep), change the tmp dir configuration. For this, create two variables in the config file: 'local_scratch' and 'local_scratch_env'. The first one is a path a the latest a system env variable. Then in the snakemake file, try to read the env option first, if fails, read the path, else: don't copy the file (probably set to None). Therefore, make the file copying optional.
[x] For the epoch pipeline (before merging with iEEGPrep). There's the chance that storage gets full if multiple patients are copying the edf file to tmp dir. To solve this potential issue, create a new resources variable (something like 'tmpdir_size') under 'Resources' for the epoch rule. Then, the user SHOULD indicate the total amount of bytes available through the CLI using --resources (Command line interface — Snakemake 7.25.0 documentation). This will make snakemake to parallelize as many subjects or iterations as possible without saturating the memory.
[x] Put all the preprocessing steps + the epoch rule into one group.
[x] Run benchmark to figure out the resources taken by each rule and make the parallelization more efficient. Extra: You could also use group components from the CLI (Command line interface — Snakemake 7.25.0 documentation), which just, instead of submitting one job per subject), can submit one job per each n subjects (it's on you to figure out the n number). However, this is something on the CLI so it would be just for me, possibly add a note about it in the documentation.