baselime / helm-charts

Helm Charts for Baselime
MIT License
2 stars 0 forks source link

Fluent-bit host is incorrect #2

Closed tunamaguro closed 4 months ago

tunamaguro commented 4 months ago

Problem Description

I followed this documentation and added my API key to the custom-values.yaml file. Then, I installed the Helm chart using the following commands:

helm repo add baselime-logs-exporter https://baselime.github.io/helm-charts
helm repo update
helm install baselime-logs-exporter baselime-logs-exporter/baselime-logs-exporter --values custom-values.yaml

However, when checking the logs, I noticed a 401 error:

$ kubectl -n baselime logs baselime-collector-cmqvz -f
Fluent Bit v3.0.3
* Copyright (C) 2015-2024 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

...

[2024/04/30 06:25:23] [error] [output:http:http.0] events.baselime.cc:443, HTTP status=401
{"message":"Unauthorised"}
[2024/04/30 06:25:23] [ warn] [engine] failed to flush chunk '1-1714458293.874966719.flb', retry in 8 seconds: task_id=0, input=tail.0 > output=http.0 (out_id=0)
[2024/04/30 06:25:32] [error] [output:http:http.0] events.baselime.cc:443, HTTP status=401
{"message":"Unauthorised"}
[2024/04/30 06:25:32] [error] [engine] chunk '1-1714458293.874966719.flb' cannot be retried: task_id=0, input=tail.0 > output=http.0
[2024/04/30 06:25:53] [error] [output:http:http.0] events.baselime.cc:443, HTTP status=401
{"message":"Unauthorised"}
[2024/04/30 06:25:53] [ warn] [engine] failed to flush chunk '1-1714458327.412361089.flb', retry in 7 seconds: task_id=0, input=tail.0 > output=http.0 (out_id=0)
[2024/04/30 06:26:00] [error] [output:http:http.0] events.baselime.cc:443, HTTP status=401
{"message":"Unauthorised"}
...

Expected Behavior

Fluent Bit should send log data without any errors or authentication issues.

Potential Cause

Upon further investigation, I found that the issue might be caused by an incorrect destination host in the Fluent Bit configuration. In the current fluentbit-configmap.yaml file, the destination host is set to events.baselime.cc

    [OUTPUT]
      Name         http
      Host         events.baselime.cc
      Port         443
      URI          /v1/logs

https://github.com/baselime/helm-charts/blob/main/charts/baselime-logs-exporter/templates/fluentbit-configmap.yaml#L33

However, according to this documentation, the correct destination host should be events.baselime.io

curl -X 'POST' 'https://events.baselime.io/v1/logs' \
  -H 'x-api-key: $BASELIME_API_KEY' \
  -H 'Content-Type: application/json' \
  -H 'x-service: my-service' \
  -d '[
        {
          "message": "This is an example log event",
          "error": "TypeError: Cannot read property 'something' of undefined",
          "requestId": "6092d6f0-3bfa-4d62-9d0b-5bc7ae6518a1",
          "namespace": "https://api.domain.com/resource/{id}"
        },
        {
          "message": "This is another example log event",
          "requestId": "6092d6f0-3bfa-4d62-9d0b-5bc7ae6518a1",
          "data": {"userId": "01HBRCB38K2K4V5SDR7YC1D0ZB"},
          "duration": 127
        }
      ]'

Temporary Workaround

I applied the manifest with the modified Host using helm template and it worked.

+       Host         events.baselime.io
-       Host         events.baselime.cc
$ kubectl logs -f baselime-collector-sf6z5 
Fluent Bit v3.0.3
* Copyright (C) 2015-2024 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

...

[2024/04/30 06:56:41] [ info] [output:http:http.0] events.baselime.io:443, HTTP status=202
{"message":"Request Accepted"}
[2024/04/30 06:57:11] [ info] [output:http:http.0] events.baselime.io:443, HTTP status=202
{"message":"Request Accepted"}
...
Lastin commented 4 months ago

Thanks for pointing it out @tunamaguro, I forgot to change it after running some tests. I've corrected it in the version 0.1.2.