search

DataDog / chaos-controller

:monkey: :fire: Datadog Failure Injection System for Kubernetes
Apache License 2.0
178 stars 28 forks source link

User Issue: CPU Pressure experiment could not inject the disruption successfully #731

Closed expFlower closed 1 year ago

expFlower commented 1 year ago

Describe the bug While executing a CPU Pressure disruption the injector fails to inject the disruption successfully.

To Reproduce Steps to reproduce the behavior:

  1. Apply the following disruption on an EKS cluster: 
    apiVersion: chaos.datadoghq.com/v1beta1
kind: Disruption
metadata:
name: cpu-pressure-pod
namespace: chaos-engineering-framework
annotations:
chaos.datadoghq.com/environment: dev
spec:
level: pod
duration: 15m
selector:
app.kubernetes.io/name: "fault-injection-showcase"
count: 100%
cpuPressure:
count: 100%
staticTargeting: true

Expected behavior The expectation is that the chaos injector finds the targets and successfully injects pressure to containers.

Screenshots The chaos-contoller correctly identifies the targets:

Name:         cpu-pressure-pod
Namespace:    chaos-engineering-framework
Labels:       <none>
Annotations:  chaos.datadoghq.com/environment: dev
API Version:  chaos.datadoghq.com/v1beta1
Kind:         Disruption
Metadata:
  Creation Timestamp:  2023-06-28T10:42:56Z
  Finalizers:
    finalizer.chaos.datadoghq.com
  Generation:  1
  Managed Fields:
    ...
Spec:
  Count:  100%
  Cpu Pressure:
    Count:   100%
  Duration:  15m0s
  Level:     pod
  Selector:
    app.kubernetes.io/name:  fault-injection-showcase
  Static Targeting:          true
  Triggers:
    Create Pods:
      Not Before:  <nil>
    Inject:
      Not Before:  <nil>
Status:
  Desired Targets Count:   4
  Ignored Targets Count:   0
  Injected Targets Count:  0
  Injection Status:        PreviouslyPartiallyInjected
  Selected Targets Count:  4
  Target Injections:
    demo-curl-66dd49f77-cwrcd:
      Injection Status:  NotInjected
    demo-nginx-b764fb599-2ncrz:
      Injection Status:  NotInjected
    fault-injection-showcase-plugin-master-template-79cfc667bd6phhg:
      Injection Status:  NotInjected
    fault-injection-showcase-plugin-master-template-79cfc667bd77b55:
      Injection Status:  NotInjected
Events:                  <none>

Pod describe:

Name:             chaos-cpu-pressure-pod-hggrb
Namespace:        chaos-engineering-framework
Priority:         0
Service Account:  chaos-injector
Node:             ip-10-72-87-212.us-west-2.compute.internal/10.72.87.212
Start Time:       Wed, 28 Jun 2023 11:42:58 +0100
Labels:           chaos.datadoghq.com/disruption-kind=cpu-pressure
                  chaos.datadoghq.com/disruption-name=cpu-pressure-pod
                  chaos.datadoghq.com/disruption-namespace=chaos-engineering-framework
                  chaos.datadoghq.com/target=demo-curl-66dd49f77-cwrcd
Annotations:      kubernetes.io/psp: eks.privileged
                  sidecar.istio.io/inject: false
Status:           Succeeded
IP:               10.72.117.226
IPs:
  IP:  10.72.117.226
Containers:
  injector:
    Container ID:  docker://c09f06f1d55d74f521ec421946beab71b14352a65cab7ef8f8d2d697736b14db
    Image:         xxxxxxx/datadog/chaos-injector:7.18.0
    Image ID:      docker-pullable://xxxxxxx/datadog/chaos-injector@sha256:28148f9f2decad765d8ef70c2d5dd9b1e1ba808741e339721e012822b2ea8743
    Port:          <none>
    Host Port:     <none>
    Args:
      cpu-pressure
      --count
      100%
      --metrics-sink
      noop
      --level
      pod
      --target-containers
      curl;docker://a7fb85a4599d94d18fa91da244a789efae96ce5bf4034126e59f551aecd94201,dummy;docker://4edab9f9b56cc00eb48e1ee298eabd6866e1b67b2bc3babccd3cbeb370e7b048,istio-proxy;docker://bd8fd379fe3eadab07e6b9f9b37788df754b47fe96d121123ce0983b13f04e4b
      --target-pod-ip
      10.72.83.133
      --chaos-namespace
      chaos-engineering-framework
      --log-context-disruption-name
      cpu-pressure-pod
      --log-context-disruption-namespace
      chaos-engineering-framework
      --log-context-target-name
      demo-curl-66dd49f77-cwrcd
      --log-context-target-node-name
      ip-10-72-87-212.us-west-2.compute.internal
      --not-injected-before
      2023-06-28T10:42:56Z
      --deadline
      2023-06-28T10:57:55Z
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Wed, 28 Jun 2023 11:42:59 +0100
      Finished:     Wed, 28 Jun 2023 11:57:55 +0100
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     0
      memory:  0
    Requests:
      cpu:      0
      memory:   0
    Readiness:  exec [cat /tmp/readiness_probe] delay=0s timeout=1s period=1s #success=1 #failure=5
    Environment:
      DD_ENTITY_ID:                         (v1:metadata.uid)
      DD_AGENT_HOST:                        (v1:status.hostIP)
      TARGET_POD_HOST_IP:                   (v1:status.hostIP)
      CHAOS_POD_IP:                         (v1:status.podIP)
      INJECTOR_POD_NAME:                   chaos-cpu-pressure-pod-hggrb (v1:metadata.name)
      CHAOS_INJECTOR_MOUNT_HOST:           /mnt/host/
      CHAOS_INJECTOR_MOUNT_PROC:           /mnt/host/proc/
      CHAOS_INJECTOR_MOUNT_SYSRQ:          /mnt/sysrq
      CHAOS_INJECTOR_MOUNT_SYSRQ_TRIGGER:  /mnt/sysrq-trigger
      CHAOS_INJECTOR_MOUNT_CGROUP:         /mnt/cgroup/
    Mounts:
      /mnt/cgroup from cgroup (rw)
      /mnt/host from host (ro)
      /mnt/sysrq from sysrq (rw)
      /mnt/sysrq-trigger from sysrq-trigger (rw)
      /run from run (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-l8lnd (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  run:
    Type:          HostPath (bare host directory volume)
    Path:          /run
    HostPathType:  Directory
  proc:
    Type:          HostPath (bare host directory volume)
    Path:          /proc
    HostPathType:  Directory
  sysrq:
    Type:          HostPath (bare host directory volume)
    Path:          /proc/sys/kernel/sysrq
    HostPathType:  File
  sysrq-trigger:
    Type:          HostPath (bare host directory volume)
    Path:          /proc/sysrq-trigger
    HostPathType:  File
  cgroup:
    Type:          HostPath (bare host directory volume)
    Path:          /sys/fs/cgroup
    HostPathType:  Directory
  host:
    Type:          HostPath (bare host directory volume)
    Path:          /
    HostPathType:  Directory
  kube-api-access-l8lnd:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              kubernetes.io/arch=amd64
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:                      <none>

AWS node docker info (Full Docker inspect collected and save if needed)

sh-4.2$ sudo docker ps | grep k8s_dummy_demo-curl-66dd49f77-cwrcd_chaos-engineering-framework_0894918f-93c6-45c3-b842-978574d0dbfe_0
4edab9f9b56c   xxxxxxxxx/ubuntu                                                                             "/bin/bash -c 'while…"   17 hours ago    Up 17 hours              k8s_dummy_demo-curl-66dd49f77-cwrcd_chaos-engineering-framework_0894918f-93c6-45c3-b842-978574d0dbfe_0

Environment:

Additional context

Injector logs:

{"level":"info","ts":1687948979446.6792,"caller":"injector/main.go:244","message":"injector targeting container","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","containerID":"docker://4edab9f9b56cc00eb48e1ee298eabd6866e1b67b2bc3babccd3cbeb370e7b048","container name":"/k8s_dummy_demo-curl-66dd49f77-cwrcd_chaos-engineering-framework_0894918f-93c6-45c3-b842-978574d0dbfe_0"}
{"level":"info","ts":1687948979449.3525,"caller":"injector/main.go:244","message":"injector targeting container","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","containerID":"docker://bd8fd379fe3eadab07e6b9f9b37788df754b47fe96d121123ce0983b13f04e4b","container name":"/k8s_istio-proxy_demo-curl-66dd49f77-cwrcd_chaos-engineering-framework_0894918f-93c6-45c3-b842-978574d0dbfe_0"}
{"level":"debug","ts":1687948979450.1873,"caller":"netns/netns.go:52","message":"Retrieved root namespace and target namespace","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","rootns":9,"targetns":10,"targetnsPath":"/mnt/host/proc/25830/ns/net"}
{"level":"debug","ts":1687948979450.2866,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"memory","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/a7fb85a4599d94d18fa91da244a789efae96ce5bf4034126e59f551aecd94201"}
{"level":"debug","ts":1687948979450.3376,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"devices","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/a7fb85a4599d94d18fa91da244a789efae96ce5bf4034126e59f551aecd94201"}
{"level":"debug","ts":1687948979450.347,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"net_cls","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/a7fb85a4599d94d18fa91da244a789efae96ce5bf4034126e59f551aecd94201"}
{"level":"debug","ts":1687948979450.3552,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"net_prio","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/a7fb85a4599d94d18fa91da244a789efae96ce5bf4034126e59f551aecd94201"}
{"level":"debug","ts":1687948979450.363,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"cpuset","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/a7fb85a4599d94d18fa91da244a789efae96ce5bf4034126e59f551aecd94201"}
{"level":"debug","ts":1687948979450.3704,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"pids","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/a7fb85a4599d94d18fa91da244a789efae96ce5bf4034126e59f551aecd94201"}
{"level":"debug","ts":1687948979450.391,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"blkio","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/a7fb85a4599d94d18fa91da244a789efae96ce5bf4034126e59f551aecd94201"}
{"level":"debug","ts":1687948979450.3997,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"perf_event","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/a7fb85a4599d94d18fa91da244a789efae96ce5bf4034126e59f551aecd94201"}
{"level":"debug","ts":1687948979450.4202,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"hugetlb","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/a7fb85a4599d94d18fa91da244a789efae96ce5bf4034126e59f551aecd94201"}
{"level":"debug","ts":1687948979450.4287,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"cpu","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/a7fb85a4599d94d18fa91da244a789efae96ce5bf4034126e59f551aecd94201"}
{"level":"debug","ts":1687948979450.4355,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"cpuacct","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/a7fb85a4599d94d18fa91da244a789efae96ce5bf4034126e59f551aecd94201"}
{"level":"debug","ts":1687948979450.4429,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"freezer","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/a7fb85a4599d94d18fa91da244a789efae96ce5bf4034126e59f551aecd94201"}
{"level":"debug","ts":1687948979450.45,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"name=systemd","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/a7fb85a4599d94d18fa91da244a789efae96ce5bf4034126e59f551aecd94201"}
{"level":"debug","ts":1687948979450.5627,"caller":"netns/netns.go:52","message":"Retrieved root namespace and target namespace","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","rootns":11,"targetns":12,"targetnsPath":"/mnt/host/proc/26062/ns/net"}
{"level":"debug","ts":1687948979450.6387,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"cpu","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/4edab9f9b56cc00eb48e1ee298eabd6866e1b67b2bc3babccd3cbeb370e7b048"}
{"level":"debug","ts":1687948979450.6533,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"freezer","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/4edab9f9b56cc00eb48e1ee298eabd6866e1b67b2bc3babccd3cbeb370e7b048"}
{"level":"debug","ts":1687948979450.661,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"net_cls","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/4edab9f9b56cc00eb48e1ee298eabd6866e1b67b2bc3babccd3cbeb370e7b048"}
{"level":"debug","ts":1687948979450.684,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"net_prio","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/4edab9f9b56cc00eb48e1ee298eabd6866e1b67b2bc3babccd3cbeb370e7b048"}
{"level":"debug","ts":1687948979450.692,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"cpuset","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/4edab9f9b56cc00eb48e1ee298eabd6866e1b67b2bc3babccd3cbeb370e7b048"}
{"level":"debug","ts":1687948979450.6987,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"pids","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/4edab9f9b56cc00eb48e1ee298eabd6866e1b67b2bc3babccd3cbeb370e7b048"}
{"level":"debug","ts":1687948979450.7134,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"memory","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/4edab9f9b56cc00eb48e1ee298eabd6866e1b67b2bc3babccd3cbeb370e7b048"}
{"level":"debug","ts":1687948979450.7344,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"blkio","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/4edab9f9b56cc00eb48e1ee298eabd6866e1b67b2bc3babccd3cbeb370e7b048"}
{"level":"debug","ts":1687948979450.742,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"perf_event","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/4edab9f9b56cc00eb48e1ee298eabd6866e1b67b2bc3babccd3cbeb370e7b048"}
{"level":"debug","ts":1687948979450.7493,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"devices","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/4edab9f9b56cc00eb48e1ee298eabd6866e1b67b2bc3babccd3cbeb370e7b048"}
{"level":"debug","ts":1687948979450.7686,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"hugetlb","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/4edab9f9b56cc00eb48e1ee298eabd6866e1b67b2bc3babccd3cbeb370e7b048"}
{"level":"debug","ts":1687948979450.7764,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"cpuacct","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/4edab9f9b56cc00eb48e1ee298eabd6866e1b67b2bc3babccd3cbeb370e7b048"}
{"level":"debug","ts":1687948979450.7834,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"name=systemd","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/4edab9f9b56cc00eb48e1ee298eabd6866e1b67b2bc3babccd3cbeb370e7b048"}
{"level":"debug","ts":1687948979450.8774,"caller":"netns/netns.go:52","message":"Retrieved root namespace and target namespace","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","rootns":13,"targetns":14,"targetnsPath":"/mnt/host/proc/25517/ns/net"}
{"level":"debug","ts":1687948979450.9482,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"blkio","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/bd8fd379fe3eadab07e6b9f9b37788df754b47fe96d121123ce0983b13f04e4b"}
{"level":"debug","ts":1687948979450.9734,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"net_cls","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/bd8fd379fe3eadab07e6b9f9b37788df754b47fe96d121123ce0983b13f04e4b"}
{"level":"debug","ts":1687948979450.9814,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"hugetlb","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/bd8fd379fe3eadab07e6b9f9b37788df754b47fe96d121123ce0983b13f04e4b"}
{"level":"debug","ts":1687948979450.9883,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"cpuacct","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/bd8fd379fe3eadab07e6b9f9b37788df754b47fe96d121123ce0983b13f04e4b"}
{"level":"debug","ts":1687948979450.9993,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"freezer","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/bd8fd379fe3eadab07e6b9f9b37788df754b47fe96d121123ce0983b13f04e4b"}
{"level":"debug","ts":1687948979451.0063,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"pids","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/bd8fd379fe3eadab07e6b9f9b37788df754b47fe96d121123ce0983b13f04e4b"}
{"level":"debug","ts":1687948979451.0137,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"memory","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/bd8fd379fe3eadab07e6b9f9b37788df754b47fe96d121123ce0983b13f04e4b"}
{"level":"debug","ts":1687948979451.0205,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"devices","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/bd8fd379fe3eadab07e6b9f9b37788df754b47fe96d121123ce0983b13f04e4b"}
{"level":"debug","ts":1687948979451.0273,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"net_prio","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/bd8fd379fe3eadab07e6b9f9b37788df754b47fe96d121123ce0983b13f04e4b"}
{"level":"debug","ts":1687948979451.0347,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"cpu","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/bd8fd379fe3eadab07e6b9f9b37788df754b47fe96d121123ce0983b13f04e4b"}
{"level":"debug","ts":1687948979451.0562,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"name=systemd","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/bd8fd379fe3eadab07e6b9f9b37788df754b47fe96d121123ce0983b13f04e4b"}
{"level":"debug","ts":1687948979451.072,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"cpuset","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/bd8fd379fe3eadab07e6b9f9b37788df754b47fe96d121123ce0983b13f04e4b"}
{"level":"debug","ts":1687948979451.0916,"caller":"cgroup/manager_linux.go:37","message":"adding cgroup subsystem path to manager","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","subsystem":"perf_event","path":"/kubepods/burstable/pod0894918f-93c6-45c3-b842-978574d0dbfe/bd8fd379fe3eadab07e6b9f9b37788df754b47fe96d121123ce0983b13f04e4b"}
{"level":"info","ts":1687948979451.3286,"caller":"injector/main.go:498","message":"waiting for synchronized start to begin","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","timeUntilNotInjectedBefore":"-3.451324844s"}
{"level":"info","ts":1687948979451.4324,"caller":"injector/main.go:523","message":"injecting the disruption","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","kind":"cpu-pressure"}
{"level":"info","ts":1687948979451.4646,"caller":"injector/cpu_pressure.go:55","message":"creating processes to stress target","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","targetLevel":"pod","target":"/k8s_curl_demo-curl-66dd49f77-cwrcd_chaos-engineering-framework_0894918f-93c6-45c3-b842-978574d0dbfe_0","count":"100%"}
{"level":"info","ts":1687948979451.4963,"caller":"injector/cpu_pressure.go:74","message":"percentage calculated from percentage","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","targetLevel":"pod","target":"/k8s_curl_demo-curl-66dd49f77-cwrcd_chaos-engineering-framework_0894918f-93c6-45c3-b842-978574d0dbfe_0","provided_value":"100%","percentage":100}
{"level":"debug","ts":1687948979451.5222,"caller":"noop/noop.go:41","message":"NOOP: MetricInjected false\n","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal"}
{"level":"error","ts":1687948979451.549,"caller":"injector/main.go:356","message":"disruption injection failed","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","error":"unable to create new process definition for injector: targeted container does not exists: /k8s_curl_demo-curl-66dd49f77-cwrcd_chaos-engineering-framework_0894918f-93c6-45c3-b842-978574d0dbfe_0","stacktrace":"main.inject\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/cli/injector/main.go:356\nmain.injectAndWait\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/cli/injector/main.go:525\ngithub.com/spf13/cobra.(*Command).execute\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/vendor/github.com/spf13/cobra/command.go:944\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/vendor/github.com/spf13/cobra/command.go:1068\ngithub.com/spf13/cobra.(*Command).Execute\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/vendor/github.com/spf13/cobra/command.go:992\nmain.main\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/cli/injector/main.go:134\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250"}
{"level":"info","ts":1687948979451.5876,"caller":"injector/cpu_pressure.go:55","message":"creating processes to stress target","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","targetLevel":"pod","target":"/k8s_dummy_demo-curl-66dd49f77-cwrcd_chaos-engineering-framework_0894918f-93c6-45c3-b842-978574d0dbfe_0","count":"100%"}
{"level":"info","ts":1687948979451.595,"caller":"injector/cpu_pressure.go:74","message":"percentage calculated from percentage","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","targetLevel":"pod","target":"/k8s_dummy_demo-curl-66dd49f77-cwrcd_chaos-engineering-framework_0894918f-93c6-45c3-b842-978574d0dbfe_0","provided_value":"100%","percentage":100}
{"level":"debug","ts":1687948979451.6191,"caller":"noop/noop.go:41","message":"NOOP: MetricInjected false\n","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal"}
{"level":"error","ts":1687948979451.6248,"caller":"injector/main.go:356","message":"disruption injection failed","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","error":"unable to create new process definition for injector: targeted container does not exists: /k8s_dummy_demo-curl-66dd49f77-cwrcd_chaos-engineering-framework_0894918f-93c6-45c3-b842-978574d0dbfe_0","stacktrace":"main.inject\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/cli/injector/main.go:356\nmain.injectAndWait\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/cli/injector/main.go:525\ngithub.com/spf13/cobra.(*Command).execute\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/vendor/github.com/spf13/cobra/command.go:944\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/vendor/github.com/spf13/cobra/command.go:1068\ngithub.com/spf13/cobra.(*Command).Execute\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/vendor/github.com/spf13/cobra/command.go:992\nmain.main\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/cli/injector/main.go:134\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250"}
{"level":"info","ts":1687948979451.6455,"caller":"injector/cpu_pressure.go:55","message":"creating processes to stress target","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","targetLevel":"pod","target":"/k8s_istio-proxy_demo-curl-66dd49f77-cwrcd_chaos-engineering-framework_0894918f-93c6-45c3-b842-978574d0dbfe_0","count":"100%"}
{"level":"info","ts":1687948979451.6523,"caller":"injector/cpu_pressure.go:74","message":"percentage calculated from percentage","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","targetLevel":"pod","target":"/k8s_istio-proxy_demo-curl-66dd49f77-cwrcd_chaos-engineering-framework_0894918f-93c6-45c3-b842-978574d0dbfe_0","provided_value":"100%","percentage":100}
{"level":"debug","ts":1687948979451.6636,"caller":"noop/noop.go:41","message":"NOOP: MetricInjected false\n","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal"}
{"level":"error","ts":1687948979451.669,"caller":"injector/main.go:356","message":"disruption injection failed","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","error":"unable to create new process definition for injector: targeted container does not exists: /k8s_istio-proxy_demo-curl-66dd49f77-cwrcd_chaos-engineering-framework_0894918f-93c6-45c3-b842-978574d0dbfe_0","stacktrace":"main.inject\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/cli/injector/main.go:356\nmain.injectAndWait\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/cli/injector/main.go:525\ngithub.com/spf13/cobra.(*Command).execute\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/vendor/github.com/spf13/cobra/command.go:944\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/vendor/github.com/spf13/cobra/command.go:1068\ngithub.com/spf13/cobra.(*Command).Execute\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/vendor/github.com/spf13/cobra/command.go:992\nmain.main\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/cli/injector/main.go:134\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250"}
{"level":"error","ts":1687948979451.7002,"caller":"injector/main.go:375","message":"an injector could not inject the disruption successfully, please look at the logs above for more details","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","stacktrace":"main.inject\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/cli/injector/main.go:375\nmain.injectAndWait\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/cli/injector/main.go:525\ngithub.com/spf13/cobra.(*Command).execute\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/vendor/github.com/spf13/cobra/command.go:944\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/vendor/github.com/spf13/cobra/command.go:1068\ngithub.com/spf13/cobra.(*Command).Execute\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/vendor/github.com/spf13/cobra/command.go:992\nmain.main\n\t/go/src/github.com/ujyL7oZF/0/DataDog/chaos-controller/cli/injector/main.go:134\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250"}
{"level":"warn","ts":1687948979451.7363,"caller":"injector/main.go:644","message":"waiting for system signals...","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal"}
{"level":"info","ts":1687949875000.3848,"caller":"injector/main.go:661","message":"disruption duration has expired, exiting","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal"}
{"level":"info","ts":1687949875000.5095,"caller":"injector/main.go:443","message":"disruption cpu-pressure cleaned","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal"}
{"level":"debug","ts":1687949875000.5195,"caller":"noop/noop.go:55","message":"NOOP: MetricCleaned true\n","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal"}
{"level":"info","ts":1687949875000.5322,"caller":"injector/main.go:443","message":"disruption cpu-pressure cleaned","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal"}
{"level":"debug","ts":1687949875000.5383,"caller":"noop/noop.go:55","message":"NOOP: MetricCleaned true\n","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal"}
{"level":"info","ts":1687949875000.5452,"caller":"injector/main.go:443","message":"disruption cpu-pressure cleaned","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal"}
{"level":"debug","ts":1687949875000.5557,"caller":"noop/noop.go:55","message":"NOOP: MetricCleaned true\n","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal"}
{"level":"info","ts":1687949875056.8525,"caller":"injector/main.go:828","message":"disruption(s) cleaned, now exiting","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal"}
{"level":"info","ts":1687949875056.9182,"caller":"injector/main.go:126","message":"closing metrics sink client before exiting","disruptionName":"cpu-pressure-pod","disruptionNamespace":"chaos-engineering-framework","targetName":"demo-curl-66dd49f77-cwrcd","targetNodeName":"ip-10-72-87-212.us-west-2.compute.internal","sink":"noop"}
luphaz commented 1 year ago

Hello 👋

Indeed we have identified that our docker container runtime implementation of name retrieval does not returns the expected information.

PR is in progress to fix this here

expFlower commented 1 year ago

Thanks for the rapid response @luphaz, will watch the fix progress. Please let us know if we can assist, thanks.

luphaz commented 1 year ago

I will create a release 7.20.1 with the associated fix.

I can't guarantee it will be enough to fix the issue you are encountering as you are using an environment we are not used to disrupt (EKS with docker container runtime engine)

Feel free to reopen this issue if it does not solve it.

I will ask for more informations if it's the case at that time.

luphaz commented 1 year ago

New version is released and container images are available, let us know if it helped

expFlower commented 1 year ago

Hi @luphaz, i'm out on vacation next week so short on time but that said i've deployed v7.22.0 to our environment and i've run a quick CPU Pressure disruption, this has successfully applied pressure to the targets. I'll be doing a full validation upon return but don't anticipate issues. Thanks again for the rapid turn around 🚀