search

MathMarEcol / pdyer_aus_bio

GNU General Public License v3.0
0 stars 0 forks source link

Need to be able to set worker resources easily #9

Closed PhDyellow closed 3 years ago

PhDyellow commented 3 years ago

Currently, worker resources have defaults set in the template, and custom values added in configure_parallel.R. The custom values are hard coded though, and need a git commit to change them.

A few options:

  1. Use environmental variables: Can be added anywhere in the execution chain, all the way up to the CLI used to submit the job. However, lower stages override higher stages, and higher stages need to be properly exported to propagate down.
  2. Set the variable in the Rscript line, just before tar_make_clustermq: still needs a git commit to change
  3. Just use a git commit

I am going to go with 1., and add code that will propagate env vars as needed.

PhDyellow commented 3 years ago

Solved by 5b569c89d800a4a7d0d7ba262bc8ad7bb506185d