apache / openwhisk

Apache OpenWhisk is an open source serverless cloud platform
https://openwhisk.apache.org/
Apache License 2.0
6.53k stars 1.17k forks source link

Invoker limits should not be hard-coded. #981

Open domdom82 opened 8 years ago

domdom82 commented 8 years ago

When limiting actions in terms of nproc, nfile and cgroup memory usage we currently hard-code those limits.

They are defined in

ContainerUtils.scala
MemoryLimit.scala

I propose we do not hard-code those limits. They should be adjustable via config.

My suggestion:

  1. Put them in ansible/group_vars/all
  2. Write them to whisk.properties
  3. Have Invoker read them from consul.

We should also think about making the action time limit configurable as well.

rabbah commented 8 years ago

we added some of these to a configuration file during the last round of limit enforcement and then pulled them back out for this reason: Many of these limits should be tied to entitlement not a configuration file.

domdom82 commented 8 years ago

@rabbah I see. In this case these limits can not be enforced by AppArmor because we can't make new profiles dynamically based on entitlement. The cpu/mem/disk limits will then only be handled by the invoker based on entitlement. However there should be a default set of limits.

This is a design discussion, is there a discussion issue opened for this?

perryibm commented 8 years ago

There is another parameter, however, that will fit with the proposed model and I will try to do it via consul as suggested. This is the cpu share parameter and possibly whether invoker should use parallel operations.

kirang20 commented 6 years ago

We use a self-hosted Openwhisk for running custom python functions in our application. Is there a way to increase the memory limit of actions? We have a use case which requires the actions to have more than 512MB memory.

rabbah commented 6 years ago

The memory limits for actions were factored out so you can set them per deployment. Here’s the pr https://github.com/apache/incubator-openwhisk/pull/3148

kirang20 commented 6 years ago

Thanks, @rabbah.