search

chaosblade-io / chaosblade

An easy to use and powerful chaos engineering experiment toolkit.（阿里巴巴开源的一款简单易用、功能强大的混沌实验注入工具）
https://chaosblade.io
Apache License 2.0
6k stars 950 forks source link

"cpu-percent" can not take effect in chaosblade.pod-cpu.fullload #801

Open chunli-lab opened 2 years ago

chunli-lab commented 2 years ago

Issue Description

Type: bug report , feature request or question

Describe what happened (or what feature you want)

In this case : chaosblade.pod-cpu.fullload, We set the "cpu-percent=60%", but the target object's CPU is full directly, the parameter "cpu-percent=60%" does not take effect.

Describe what you expected to happen

We want the pod's cpu can be 60%, not 100%.

How to reproduce it (as minimally and precisely as possible)

  1. The command is: "blade create k8s pod-cpu fullload --names * --cpu-percent 60 --namespace --timeout 900 --kubeconfig ~/.kube/config"

  2. Watch the change of the pod cpu.

Tell us your environment

k8s pod

Anything else we need to know?

ntlzthm8 commented 1 year ago

Confirm.

k8s container-cpu load
k8s pod-cpu load

not effect.

kublet version - v1.22.17 CONTAINER-RUNTIME - docker://23.0.2

after 5-7 second then experiment is started, pod is restarting, but cpu not loaded

image

operator log

time="2023-04-01T18:22:42Z" level=info msg="start to create experiment" experiment=84d16e2df4a616bc time="2023-04-01T18:22:42Z" level=info msg="get pods by labels app=cartservice, len is 2" experiment=84d16e2df4a616bc time="2023-04-01T18:22:42Z" level=info msg="set container list: [{Id: ContainerRuntime:docker ContainerId:20c956ffa24f ContainerName:server PodName:cartservice-7584dccf-5twfm NodeName:k8s-node1-vm.central1.internal Namespace:default} {Id: ContainerRuntime:docker ContainerId:cfac18fa9cde ContainerName:server PodName:cartservice-7584dccf-ndkl2 NodeName:k8s-node2-vm.central1.internal Namespace:default}]" experiment=84d16e2df4a616bc time="2023-04-01T18:22:42Z" level=info msg="start to execute: &{Target:cpu Scope:container ActionName:fullload ActionFlags:map[cgroup-root:/host-sys/fs/cgroup chaosblade-deploy-mode: chaosblade-download-url: chaosblade-path:/opt climb-time: container-ids: container-index:0 container-names: cpu-count: cpu-index: cpu-list: cpu-percent:50 endpoint: evict-count: evict-group: evict-percent: labels:app=cartservice names: namespace:default timeout: uid:] ActionPrograms:[] ActionCategories:[] ActionProcessHang:false}" experiment=84d16e2df4a616bc time="2023-04-01T18:22:42Z" level=info msg="experiment identifiers: [{{ docker 20c956ffa24f server cartservice-7584dccf-5twfm k8s-node1-vm.central1.internal default} /opt/chaosblade/blade create cri cpu fullload --cpu-percent=50 --cgroup-root=/host-sys/fs/cgroup --container-id 20c956ffa24f --container-runtime docker 0 chaosblade-tool-gksfg chaosblade chaosblade-tool} {{ docker cfac18fa9cde server cartservice-7584dccf-ndkl2 k8s-node2-vm.central1.internal default} /opt/chaosblade/blade create cri cpu fullload --cpu-percent=50 --cgroup-root=/host-sys/fs/cgroup --container-id cfac18fa9cde --container-runtime docker 0 chaosblade-tool-q5gl6 chaosblade chaosblade-tool}]" experiment=84d16e2df4a616bc time="2023-04-01T18:22:42Z" level=info msg="execute identifier: {ContainerObjectMeta:{Id: ContainerRuntime:docker ContainerId:20c956ffa24f ContainerName:server PodName:cartservice-7584dccf-5twfm NodeName:k8s-node1-vm.central1.internal Namespace:default} Command:/opt/chaosblade/blade create cri cpu fullload --cpu-percent=50 --cgroup-root=/host-sys/fs/cgroup --container-id 20c956ffa24f --container-runtime docker Error: Code:0 ChaosBladePodName:chaosblade-tool-gksfg ChaosBladeNamespace:chaosblade ChaosBladeContainerName:chaosblade-tool}" experiment=84d16e2df4a616bc time="2023-04-01T18:22:42Z" level=info msg="Exec command in pod" command="[/opt/chaosblade/blade create cri cpu fullload --cpu-percent=50 --cgroup-root=/host-sys/fs/cgroup --container-id 20c956ffa24f --container-runtime docker]" container=chaosblade-tool podName=chaosblade-tool-gksfg podNamespace=chaosblade time="2023-04-01T18:22:42Z" level=info msg="execute identifier: {ContainerObjectMeta:{Id: ContainerRuntime:docker ContainerId:cfac18fa9cde ContainerName:server PodName:cartservice-7584dccf-ndkl2 NodeName:k8s-node2-vm.central1.internal Namespace:default} Command:/opt/chaosblade/blade create cri cpu fullload --cpu-percent=50 --cgroup-root=/host-sys/fs/cgroup --container-id cfac18fa9cde --container-runtime docker Error: Code:0 ChaosBladePodName:chaosblade-tool-q5gl6 ChaosBladeNamespace:chaosblade ChaosBladeContainerName:chaosblade-tool}" experiment=84d16e2df4a616bc time="2023-04-01T18:22:42Z" level=info msg="Exec command in pod" command="[/opt/chaosblade/blade create cri cpu fullload --cpu-percent=50 --cgroup-root=/host-sys/fs/cgroup --container-id cfac18fa9cde --container-runtime docker]" container=chaosblade-tool podName=chaosblade-tool-q5gl6 podNamespace=chaosblade time="2023-04-01T18:22:42Z" level=info msg="get output message" command="[/opt/chaosblade/blade create cri cpu fullload --cpu-percent=50 --cgroup-root=/host-sys/fs/cgroup --container-id cfac18fa9cde --container-runtime docker]" container=chaosblade-tool err= out="{\"code\":200,\"success\":true,\"result\":\"bcced17eac159069\"}" podName=chaosblade-tool-q5gl6 podNamespace=chaosblade time="2023-04-01T18:22:42Z" level=info msg="exec output: {\"code\":200,\"success\":true,\"result\":\"bcced17eac159069\"}\n" location=github.com/chaosblade-io/chaosblade-spec-go/util.Infof uid= time="2023-04-01T18:22:42Z" level=info msg="get output message" command="[/opt/chaosblade/blade create cri cpu fullload --cpu-percent=50 --cgroup-root=/host-sys/fs/cgroup --container-id 20c956ffa24f --container-runtime docker]" container=chaosblade-tool err= out="{\"code\":200,\"success\":true,\"result\":\"1061c4e6526d3e80\"}" podName=chaosblade-tool-gksfg podNamespace=chaosblade time="2023-04-01T18:22:42Z" level=info msg="exec output: {\"code\":200,\"success\":true,\"result\":\"1061c4e6526d3e80\"}\n" location=github.com/chaosblade-io/chaosblade-spec-go/util.Infof uid= time="2023-04-01T18:22:42Z" level=info msg="success: true, statuses: [{Id:bcced17eac159069 State:Success Code:0 Error: Success:true Kind:container Identifier:default/k8s-node2-vm.central1.internal/cartservice-7584dccf-ndkl2/server/cfac18fa9cde/docker} {Id:1061c4e6526d3e80 State:Success Code:0 Error: Success:true Kind:container Identifier:default/k8s-node1-vm.central1.internal/cartservice-7584dccf-5twfm/server/20c956ffa24f/docker}]" experiment=84d16e2df4a616bc time="2023-04-01T18:22:42Z" level=info msg="trigger update event, name: 84d16e2df4a616bc"