search

reanahub / reana

REANA: Reusable research data analysis platform
https://docs.reana.io
MIT License
127 stars 54 forks source link

helm: configurable quota accounting behaviour #562

Closed tiborsimko closed 3 years ago

tiborsimko commented 3 years ago

Current behaviour

When somebody installs 0.8.0-alpha.4, by default the quota system is not imposing any limits on users:

$ reana-client quota-show --resource cpu --report limit   
No limit.
$ reana-client quota-show --resource disk --report limit
No limit.

However, when a user runs a few workflows, the quota consumption is being calculated:

$ reana-client quota-show --resource cpu --report usage
60328
$ reana-client quota-show --resource disk --report usage   
1577932

This causes troubles with workflow runtime pod termination, see https://github.com/reanahub/reana-db/issues/149.

As an administrator, if you would like to deploy a system that does not care about quotas, you currently cannot without wild patches such as https://github.com/reanahub/reana-db/pull/150#discussion_r733896652

Expected behaviour

It should be possible to deploy a REANA system easily for the use case of no limits, no quota calculations. This means to introduce a new easy-to-modify helm value that could alter quota usage calculation policy, e.g. do it after every workflow run (i.e. current behaviour), e.g. do it nightly (i.e. via crontab-like updater, as we do now; but not as part of termination tasks, which is not scalable); e.g. don't do it at all (i.e. completely turned off, for the fastest termination tasks, as in the wild patch above). Note that various policies might also affect how quota usage is being displayed (or not) in the web UI, for the sake of consistency.

See also https://github.com/reanahub/reana-db/issues/149

VMois commented 3 years ago

Another related discussion regarding how to nicely handle env variable for quota updates.