search

humio / humio-helm-charts

Helm Charts for Humio Components
Apache License 2.0
9 stars 17 forks source link

getaddrinfo(host='cloud.humio.com:443'): Name or service not known #137

Open gregbird101 opened 2 years ago

gregbird101 commented 2 years ago

Situation: I have followed the instructions to launch the helm chart and I get the below error Background: Followed these instructions https://library.humio.com/stable/docs/ingesting-data/log-formats/kubernetes/index.html Assessment: It seems that the fluentbit container is unable to resolve DNS however loading up a DNS utils pod in the same namespace it can Recommendation: I have tried IP the cloud humio IP addresses and other enpoint options https://library.humio.com/reference/endpoints/#endpoints

Any help would be welcome - is it possible to launch the helm chart with the fluentbit debug pod to allow shell or bash?

[2022/01/31 09:50:47] [ warn] net_tcp_fd_connect: getaddrinfo(host='cloud.humio.com:443'): Name or service not known
[2022/01/31 09:50:47] [ warn] net_tcp_fd_connect: getaddrinfo(host='cloud.humio.com:443'): Name or service not known
[2022/01/31 09:50:47] [ warn] net_tcp_fd_connect: getaddrinfo(host='cloud.humio.com:443'): Name or service not known
[2022/01/31 09:50:47] [error] [io_tls] flb_io_tls.c:359 NET - Connection was reset by peer
[2022/01/31 09:50:47] [error] [io_tls] flb_io_tls.c:359 NET - Connection was reset by peer
[2022/01/31 09:50:47] [error] [io_tls] flb_io_tls.c:359 NET - Connection was reset by peer

Actions: Setup helm

helm repo add humio https://humio.github.io/humio-helm-charts
helm repo update

Create file: humio-agent.yml

humio-fluentbit:
  enabled: true
  humioHostname: cloud.humio.com:9200
  es:
    tls: true

Setup Helm

helm install humio humio/humio-helm-charts \
  --namespace logging \
  --set humio-fluentbit.token=REDACTED \
  --values humio-agent.yml

Create DNS utils

apiVersion: v1
kind: Pod
metadata:
  name: dnsutils
  namespace: logging
spec:
  containers:
  - name: dnsutils
    image: k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.3
    command:
      - sleep
      - "3600"
    imagePullPolicy: IfNotPresent
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"
  restartPolicy: Always

Check all DNS is working

kubectl exec -n logging -i -t dnsutils -- nslookup cloud.humio.com
kubectl exec -n logging -ti dnsutils -- cat /etc/resolv.conf
kubectl get pods --namespace=kube-system -l k8s-app=kube-dns
kubectl get svc --namespace=kube-system 
kubectl get endpoints kube-dns --namespace=kube-system

--- Update ---

------- PARTIAL SOLVE ------------- edit the values.yaml humio-helm/humio-helm-charts/charts/humio-fluentbit/values.yaml

outputConfig: |-
  [OUTPUT]
      Name  es
      Match *
      Host cloud.humio.com <=========== HARDCODED
      Port 443 <==================== HARDCODED
      tls ${FLUENT_ELASTICSEARCH_TLS}
      tls.verify true <================= HARDCODED
      HTTP_User test123
      HTTP_Passwd ${HUMIO_INGEST_TOKEN}
      Logstash_Format On
      Retry_Limit False
      Type  flb_type
      Time_Key @timestamp
      Replace_Dots On
      Logstash_Prefix FluentBitHelmChart
      Buffer_Size 5MB

helm upgrade humio ./humio-helm/humio-helm-charts --set humio-fluentbit.token=REDACTED --namespace logging --values humio-agent.yml I get data in humio syslog-k8s: 23423 errors 23423 events

------- SOLVE ------------ Its to do with the :433

THIS BREAKS

humio-fluentbit:
  enabled: true
  humioHostname: cloud.humio.com:443
  es:
    tls: true

THIS WORKS

humio-fluentbit:
  enabled: true
  humioHostname: cloud.humio.com
  es:
    tls: true

Detail: Where i fell down is in the documentation "Replace $YOUR_HUMIO_URL with the hostname of your Humio installation." in here It links straight to endpoints Which has both URL and PORT.

Would it be possible to add an example of the above?

Also further Options could be that we split out the URL and port so when its passed to the values.yml it doesn't get evaluated as one?