Upgrade from 0.15.0 or fresh install dynatrace-operator 1.0.0: oneagent unable to start container process

[dhorner71]

dynatrace support ticket #306247

Describe the bug upgrading from 0.15.0 to 1.0.0 with helm or fresh install produces oneagent daemonset pod warnings:

pod event: OCI runtime exec failed: exec failed: unable to start container process: exec: "/usr/bin/watchdog-healthcheck64": stat /usr/bin/watchdog-healthcheck64: no such file or directory: unknown

pod status: type: Ready status: 'False' reason: ContainersNotReady message: 'containers with unready status: [dynatrace-oneagent]'

To Reproduce Steps to reproduce the behavior:

1. either fresh install or upgrade from 0.15.0 helm chart

helm sets: apiUrl = apiToken = dataIngestToken = installDRD = true webhook.hostNetwork = true image = customPullSecret = csidriver.enabled = false

dynakube 1.0.0: apiVersion: dynatrace.com/v1beta1 kind: DynaKube metadata: annotations: feature.dynatrace.com/automatic-kubernetes-api-monitoring: 'true' name: namespace: dynatrace spec: activeGate: capabilities:

• routing
• kubernetes-monitoring
• dynatrace-api
• metrics-ingest group: aws-eks image: /docker/dynatrace/linux/activegate:latest resources: limits: cpu: 1000m memory: 1.5Gi requests: cpu: 500m memory: 512Mi apiUrl: https://.live.dynatrace.com/api customPullSecret: networkZone: oneAgent: classicFullStack: args:
  • '--set-host-group=' env:
  • name: ONEAGENT_ENABLE_VOLUME_STORAGE value: 'false' image: /docker/dynatrace/linux/oneagent:latest tolerations:
  • effect: NoSchedule key: node-role.kubernetes.io/master operator: Exists
  • effect: NoSchedule key: node-role.kubernetes.io/control-plane operator: Exists skipCertCheck: true

dynakube 0.15.0: apiVersion: dynatrace.com/v1beta1 kind: DynaKube metadata: annotations: feature.dynatrace.com/automatic-kubernetes-api-monitoring: 'true' name: namespace: dynatrace spec: activeGate: capabilities:

• routing
• kubernetes-monitoring
• dynatrace-api
• metrics-ingest group: image: /docker/dynatrace/linux/activegate:latest resources: limits: cpu: 1000m memory: 1.5Gi requests: cpu: 500m memory: 512Mi apiUrl: https://.live.dynatrace.com/api customPullSecret: networkZone: oneAgent: classicFullStack: args:
  • '--set-host-group=' env:
  • name: ONEAGENT_INSTALLER_SCRIPT_URL value: https://.live.dynatrace.com/api/v1/deployment/installer/agent/unix/default/latest?arch=x86
  • name: ONEAGENT_INSTALLER_DOWNLOAD_TOKEN value:
  • name: ONEAGENT_INSTALLER_SKIP_CERT_CHECK value: 'true' image: /docker/dynatrace/linux/oneagent:latest oneAgentResources: limits: cpu: 300m memory: 1.5Gi requests: cpu: 100m memory: 512Mi tolerations:
  • effect: NoSchedule key: node-role.kubernetes.io/master operator: Exists
  • effect: NoSchedule key: node-role.kubernetes.io/control-plane operator: Exists skipCertCheck: true

Expected behavior after helm install, either upgrade or fresh, healthy pods reported with no warnings reported just like 0.15.0

Screenshots n/a

Environment (please complete the following information):

• EKS/Kubernetes 1.27
• terraform provider hashicorp/helm v2.13.0
• helm v3.9.4
• dynatrace-operator 1.0.0 helm chart
• oneagent imageID oneagent@sha256:35b90670a961e5fa932cbee14a87b60d6e583bb9883bb05dfbbe7e180f7131b2
  • activegate imageID activegate@sha256:2a1277acacfc81dc7e1a39bf24577c900026def26b740e61013549a0d17ddc0e

Additional context -probably unimportant but with EKS autoscaler, version 1.0.0 does require an additional node (4 total) from 0.15.0 (3 total) -could be related but even using latest tag on images and helm chart still produces dashboard warning: "The ActiveGate monitoring this Kubernetes cluster is outdated. Please make sure that all ActiveGates have [a minimum version of 1.279] to get the latest enhancements in Kubernetes monitoring."

• used these links as supporint info:https://docs.dynatrace.com/docs/whats-new/release-notes/dynatrace-operator/dto-fix-1-0-0#upgrade-from-dynatrace-operator-version-0-15-0 https://docs.dynatrace.com/docs/whats-new/release-notes/dynatrace-operator/dto-fix-1-0-0

[github-actions[bot]]

Thank you for opening a Dynatrace Operator Issue. We've identified and tagged the issue as a "Support request".

Dynatrace responds to requests like these via Dynatrace ONE support rather than Github. This helps our team respond as quickly as possible using the support team's tools and procedures.

Thanks for your help!