reanahub / reana-demo-atlas-recast

REANA example - RECAST analysis based on ATLAS software stack
MIT License
4 stars 33 forks source link

workflow: add kubernetes_memory_limit #32

Closed tiborsimko closed 3 years ago

tiborsimko commented 3 years ago

Enrich example with kubernetes_memory_limit directive in order to be able to run the workflow with as little memory consumption as possible. This means:

  1. Run the workflow as is now, and observe real memory usage via kubectl top pods or via logging into the node and using top for each job. For example, 2GiB can be consumed for one step.

  2. Amend workflow definition to introduce kubernetes_memory_limit for each step and use value observed above.

  3. Run example successfully on REANA and lower the value in order to have them as low as possible for each step.

This will allow ultimately to run many ATLAS RECAST demo workflows in parallel.

VMois commented 3 years ago

the memory limit of:

I will put 128MiB to be on the safe side (the number is still very small).