chaosblade-io / chaosblade

An easy to use and powerful chaos engineering experiment toolkit.(阿里巴巴开源的一款简单易用、功能强大的混沌实验注入工具)
https://chaosblade.io
Apache License 2.0
5.92k stars 943 forks source link

blade create mem load --mode ram is unstable #751

Open anxvi opened 2 years ago

anxvi commented 2 years ago

We use chaosblade for chaos engineering in a big company, version 1.4.0, and that's what we discovered:

Although we specify the low rate (1-16 MB/s) on high loads (above 80% RAM), the blade immediately reaches the 99% load and the process is killed. The flag --avoid-being-killed keeps it alive just 2-3 seconds more.

What we are aiming to achieve is having a stable high load at 95 and 100%. We guess that the rates are somehow being ignored.

Have you ever come up with this issue before? How can we fix this? The status is fine but ps aux shows the proccess terminates very quickly.

xcaspar commented 2 years ago

Please provide the linux system and kernel version. The avoid-being-killed parameter works on some systems, some doesn't

anxvi commented 2 years ago

Please provide the linux system and kernel version. The avoid-being-killed parameter works on some systems, some doesn't

OS is RHEL Server 7.9 and Kernel version is Linux 3.10.0-1160.42.2.e17.x86_64

anxvi commented 2 years ago

Please provide the linux system and kernel version. The avoid-being-killed parameter works on some systems, some doesn't

By the way, we checked through ps aux, the parameter actually escalates the priority of the process so it lasts a little longer than usually but still gets killed

imooni commented 2 years ago

Then it's gone?

anxvi commented 2 years ago

Then it's gone?

No, it's not