[Good First Issue] MonitoringEKS example is using outdated k8s version 1.14

[rl-abdulkarim]

Problem: The MonitoringEKS example is using k8s version 1.14 which is no longer supported by Amazon EKS.

❯ eksctl create cluster --name my-cluster2 --version 1.14
2021-05-05 00:29:51 [ℹ]  eksctl version 0.47.0
2021-05-05 00:29:51 [ℹ]  using region us-west-2
Error: invalid version, 1.14 is no longer supported, supported values: 1.15, 1.16, 1.17, 1.18, 1.19
see also: https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html

Suggested Fix: Use k8s version 1.19 (since it's the latest available version on Amazon EKS). This will require

• Fixing the README.md
• Fixing the beats/*.yaml files to support new k8s version

[jonnymccullagh]

I agree with @rl-abdulkarim the EKS example needs updated. I was following the tutorial by Elastic here but had errors such as:

resource mapping not found for name: "metricbeat" namespace: "kube-system" from "examples/MonitoringEKS/beats/metricbeat-kubernetes.yaml": no matches for kind "DaemonSet" in version "extensions/v1beta1"
ensure CRDs are installed first
error validating data: ValidationError(DaemonSet.spec): missing required field "selector" in io.k8s.api.apps.v1.DaemonSetSpec

I had to amend the metricbeat-kubernetes.yaml file changing outdated api versions:

apiVersion: extensions/v1beta1
apiVersion: apps/v1beta1
apiVersion: rbac.authorization.k8s.io/v1beta1

to

apiVersion: apps/v1
apiVersion: rbac.authorization.k8s.io/v1

Next I had to add spec sections (as below) to the Daemonset and Deployment sections:

    spec:
      selector:
        matchLabels:
          k8s-app: metricbeat

Also in the generated secrets.yaml file I had to amend the name:

    apiVersion: v1
    kind: Secret
    metadata:
      name: beats-secrets

The generation script creates the name as: metricbeat-secrets rather than beats-secrets

[MGCreator]

I agree with @rl-abdulkarim the EKS example needs updated. I was following the tutorial by Elastic here but had errors such as:

resource mapping not found for name: "metricbeat" namespace: "kube-system" from "examples/MonitoringEKS/beats/metricbeat-kubernetes.yaml": no matches for kind "DaemonSet" in version "extensions/v1beta1"
ensure CRDs are installed first
error validating data: ValidationError(DaemonSet.spec): missing required field "selector" in io.k8s.api.apps.v1.DaemonSetSpec

I had to amend the metricbeat-kubernetes.yaml file changing outdated api versions:

apiVersion: extensions/v1beta1
apiVersion: apps/v1beta1
apiVersion: rbac.authorization.k8s.io/v1beta1

to

apiVersion: apps/v1
apiVersion: rbac.authorization.k8s.io/v1

Next I had to add spec sections (as below) to the Daemonset and Deployment sections:

    spec:
      selector:
        matchLabels:
          k8s-app: metricbeat

Also in the generated secrets.yaml file I had to amend the name:

    apiVersion: v1
    kind: Secret
    metadata:
      name: beats-secrets

The generation script creates the name as: metricbeat-secrets rather than beats-secrets

Hello @jonnymccullagh, May you tell me where exactly this should be placed, because i am bit lost.

    spec:
      selector:
        matchLabels:
          k8s-app: metricbeat

This is how my Deamonset section looks like:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: metricbeat
  namespace: kube-system
  labels:
    k8s-app: metricbeat
spec:
  template:
    metadata:
      labels:
        k8s-app: metricbeat
    spec:
      serviceAccountName: metricbeat
      terminationGracePeriodSeconds: 30
      hostNetwork: true
      dnsPolicy: ClusterFirstWithHostNet
      containers:
      - name: metricbeat
        image: docker.elastic.co/beats/metricbeat:7.6.2
        args: [
          "-c", "/etc/metricbeat.yml",
          "-e",
          "-system.hostfs=/hostfs",
        ]

This is the Deployment section:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: metricbeat
  namespace: kube-system
  labels:
    k8s-app: metricbeat
spec:
  template:
    metadata:
      labels:
        k8s-app: metricbeat
    spec:
      serviceAccountName: metricbeat
      hostNetwork: true
      dnsPolicy: ClusterFirstWithHostNet
      containers:
      - name: metricbeat
        image: docker.elastic.co/beats/metricbeat:7.6.2
        args: [
          "-c", "/etc/metricbeat.yml",
          "-e",
        ]

Thank you in advance!