search

canonical / mysql-k8s-operator

A Charmed Operator for running MySQL on Kubernetes
https://charmhub.io/mysql-k8s
Apache License 2.0
8 stars 15 forks source link

Stabilize CI tests on GH by applying RAM constraints on controller/model level (to limit each MySQL unit to 1GB max) #205

Closed taurus-forever closed 1 year ago

taurus-forever commented 1 year ago

Hi,

GH tests crashes with various reasoning often caused by lack of RAM and a lot of units running on free GitHub runners. MySQL charm code allocates innodb_buffer_pool_size to 50% of container/vm RAM (4GB in fact). Each container's RAM is 100% of GH Runner RAM (8GB each). Each VM's RAM is also 100% of host RAM (by default).

In fact on GH Runner with 8GB RAM, we are starting two MySQL clusters, with three units each, setting innodb_buffer_pool_size=4GB for each unit...

We must limit RAM to 1GB maximum for each unit. It should be more then enough for all our tests.

See details here and here.

P.S. maybe we should request charmed-kubernetes/actions-operator to set some constraints by default as most of tests do not need a lot of RAM, but looking for tests stability.

github-actions[bot] commented 1 year ago

https://warthogs.atlassian.net/browse/DPE-1656

taurus-forever commented 1 year ago

Unfortunately, it is no possible to limit Juju app RAM usago on controller bootstrap: https://bugs.launchpad.net/juju/+bug/2016242

@paulomach please consider to lock contraints on juju test model creation:

subprocess.check_output("juju set-model-constraints cores=2 mem=1G")