use dynamic resource allocation; disable timeline for now; added some…

[BioinfoTongLI]

… default parameters

[codecov-commenter]

Codecov Report

Attention: 9 lines in your changes are missing coverage. Please review.

Comparison is base (c34922d) 36.28% compared to head (0fcf094) 36.27%. Report is 25 commits behind head on dev.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## dev #96 +/- ## ========================================== - Coverage 36.28% 36.27% -0.02% ========================================== Files 14 14 Lines 722 725 +3 ========================================== + Hits 262 263 +1 - Misses 460 462 +2 ``` | [Files](https://app.codecov.io/gh/haniffalab/webatlas-pipeline/pull/96?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=haniffalab) | Coverage Δ | | |---|---|---| | [bin/integrate\_anndata.py](https://app.codecov.io/gh/haniffalab/webatlas-pipeline/pull/96?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=haniffalab#diff-YmluL2ludGVncmF0ZV9hbm5kYXRhLnB5) | `0.00% <ø> (ø)` | | | [bin/integrate\_image.py](https://app.codecov.io/gh/haniffalab/webatlas-pipeline/pull/96?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=haniffalab#diff-YmluL2ludGVncmF0ZV9pbWFnZS5weQ==) | `0.00% <ø> (ø)` | | | [bin/process\_merscope.py](https://app.codecov.io/gh/haniffalab/webatlas-pipeline/pull/96?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=haniffalab#diff-YmluL3Byb2Nlc3NfbWVyc2NvcGUucHk=) | `23.68% <ø> (ø)` | | | [bin/build\_config.py](https://app.codecov.io/gh/haniffalab/webatlas-pipeline/pull/96?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=haniffalab#diff-YmluL2J1aWxkX2NvbmZpZy5weQ==) | `0.00% <0.00%> (ø)` | | | [bin/process\_h5ad.py](https://app.codecov.io/gh/haniffalab/webatlas-pipeline/pull/96?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=haniffalab#diff-YmluL3Byb2Nlc3NfaDVhZC5weQ==) | `85.59% <0.00%> (ø)` | | | [bin/process\_spaceranger.py](https://app.codecov.io/gh/haniffalab/webatlas-pipeline/pull/96?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=haniffalab#diff-YmluL3Byb2Nlc3Nfc3BhY2VyYW5nZXIucHk=) | `54.28% <60.00%> (-0.94%)` | :arrow_down: | | [bin/process\_xenium.py](https://app.codecov.io/gh/haniffalab/webatlas-pipeline/pull/96?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=haniffalab#diff-YmluL3Byb2Nlc3NfeGVuaXVtLnB5) | `24.63% <0.00%> (ø)` | |

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[prete]

This is not how dynamic process allocation is supposed to be used:

    cpus = { 1 * task.attempt * task.attempt}
    memory = { 10.GB * task.attempt * task.attempt}

That would force process to effectively fail at least one to get the right number of resources for the 4/20G profiles. Moreover, it also hides the number of cpus/mem that's required by each process by using task.attempt**2

I highly advice having labeled processes (not named) to be able to provide generic groups of resources like this:

process {
    withLabel: small {
        cpus = 1
        memory = 4.GB
    }
    withLabel: large {
        cpus = 4
        memory = 20.GB
    }
}

...and then go back to the processes and add the right labels.

Should you want to make those 'dynamic' too you could do something like:

process {
    withLabel: small {
        cpus = 1
        memory = { 4.GB * task.attempt }
    }
    withLabel: large {
        cpus = 4
        memory = { 20.GB * task.attempt }
    }
}