Closed RitikaLaddha closed 1 month ago
Hi @edsiper Sorry I didn't know whom to tag, could you please align anyone on this question?
I am trying to achieve below flow, would be very grateful for any guidance
Logs -> Fluentbit -> OTel Collector -> Uptrace
-> Jaeger
-> etc..
@RitikaLaddha thanks for opening this issue.
The first time I see getaddrinfo() running out of memory/capacity, it seems to be related to a large number of DNS options for your remote endpoint.
can you try adding the following option inside the [SERVICE] section ?
dns.mode LEGACY
ref: https://docs.fluentbit.io/manual/administration/networking#dns-mode
Thanks for replying Sure, let me try
still getting same error
Could you try removing the http://
part of the host
?
I just tested it to be sure and it seems that fluent-bit expects the host
setting to be a regular hostname and doesn't parse URLs.
leonardo@lima-default:/Users/leonardo/Work/Calyptia/fluent-bit/build$ cat > issue_9117.yaml <<__EOF__
service:
log_level: info
pipeline:
inputs:
- name: dummy
samples: 1
outputs:
- name: opentelemetry
match: '*'
host: http://localhost
port: 9999
tls: off
__EOF__
leonardo@lima-default:/Users/leonardo/Work/Calyptia/fluent-bit/build$ ./bin/fluent-bit -c issue_9117.yaml
Fluent Bit v3.1.5
* Copyright (C) 2015-2024 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io
______ _ _ ______ _ _ _____ __
| ___| | | | | ___ (_) | |____ |/ |
| |_ | |_ _ ___ _ __ | |_ | |_/ /_| |_ __ __ / /`| |
| _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / \ \ | |
| | | | |_| | __/ | | | |_ | |_/ / | |_ \ V /.___/ /_| |_
\_| |_|\__,_|\___|_| |_|\__| \____/|_|\__| \_/ \____(_)___/
[2024/07/31 13:49:08] [ info] [fluent bit] version=3.1.5, commit=6df1f2bf7c, pid=253081
[2024/07/31 13:49:08] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2024/07/31 13:49:08] [ info] [cmetrics] version=0.9.1
[2024/07/31 13:49:08] [ info] [ctraces ] version=0.5.2
[2024/07/31 13:49:08] [ info] [input:dummy:dummy.0] initializing
[2024/07/31 13:49:08] [ info] [input:dummy:dummy.0] storage_strategy='memory' (memory only)
[2024/07/31 13:49:08] [ info] [sp] stream processor started
[2024/07/31 13:49:10] [ warn] [net] getaddrinfo(host='http://localhost', err=15): Out of memory
[2024/07/31 13:49:10] [error] [output:opentelemetry:opentelemetry.0] no upstream connections available to http://localhost:9999
[2024/07/31 13:49:10] [ warn] [engine] failed to flush chunk '253081-1722426549.431338198.flb', retry in 7 seconds: task_id=0, input=dummy.0 > output=opentelemetry.0 (out_id=0)
^C[2024/07/31 13:49:11] [engine] caught signal (SIGINT)
[2024/07/31 13:49:11] [ warn] [engine] service will shutdown in max 5 seconds
[2024/07/31 13:49:11] [ info] [input] pausing dummy.0
[2024/07/31 13:49:11] [ warn] [net] getaddrinfo(host='http://localhost', err=15): Out of memory
[2024/07/31 13:49:11] [error] [output:opentelemetry:opentelemetry.0] no upstream connections available to http://localhost:9999
[2024/07/31 13:49:11] [error] [engine] chunk '253081-1722426549.431338198.flb' cannot be retried: task_id=0, input=dummy.0 > output=opentelemetry.0
[2024/07/31 13:49:12] [ info] [engine] service has stopped (0 pending tasks)
[2024/07/31 13:49:12] [ info] [input] pausing dummy.0
leonardo@lima-default:/Users/leonardo/Work/Calyptia/fluent-bit/build$ cat > issue_9117.yaml <<__EOF__
service:
log_level: info
pipeline:
inputs:
- name: dummy
samples: 1
outputs:
- name: opentelemetry
match: '*'
host: localhost
port: 9999
tls: off
__EOF__
leonardo@lima-default:/Users/leonardo/Work/Calyptia/fluent-bit/build$ ./bin/fluent-bit -c issue_9117.yaml
Fluent Bit v3.1.5
* Copyright (C) 2015-2024 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io
______ _ _ ______ _ _ _____ __
| ___| | | | | ___ (_) | |____ |/ |
| |_ | |_ _ ___ _ __ | |_ | |_/ /_| |_ __ __ / /`| |
| _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / \ \ | |
| | | | |_| | __/ | | | |_ | |_/ / | |_ \ V /.___/ /_| |_
\_| |_|\__,_|\___|_| |_|\__| \____/|_|\__| \_/ \____(_)___/
[2024/07/31 13:49:23] [ info] [fluent bit] version=3.1.5, commit=6df1f2bf7c, pid=253091
[2024/07/31 13:49:23] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2024/07/31 13:49:23] [ info] [cmetrics] version=0.9.1
[2024/07/31 13:49:23] [ info] [ctraces ] version=0.5.2
[2024/07/31 13:49:23] [ info] [input:dummy:dummy.0] initializing
[2024/07/31 13:49:23] [ info] [input:dummy:dummy.0] storage_strategy='memory' (memory only)
[2024/07/31 13:49:23] [ info] [sp] stream processor started
[2024/07/31 13:49:24] [error] [output:opentelemetry:opentelemetry.0] no upstream connections available to localhost:9999
[2024/07/31 13:49:24] [ warn] [engine] failed to flush chunk '253091-1722426563.435245064.flb', retry in 10 seconds: task_id=0, input=dummy.0 > output=opentelemetry.0 (out_id=0)
^C[2024/07/31 13:49:25] [engine] caught signal (SIGINT)
[2024/07/31 13:49:25] [ warn] [engine] service will shutdown in max 5 seconds
[2024/07/31 13:49:25] [ info] [input] pausing dummy.0
[2024/07/31 13:49:25] [error] [output:opentelemetry:opentelemetry.0] no upstream connections available to localhost:9999
[2024/07/31 13:49:25] [error] [engine] chunk '253091-1722426563.435245064.flb' cannot be retried: task_id=0, input=dummy.0 > output=opentelemetry.0
[2024/07/31 13:49:25] [ info] [engine] service has stopped (0 pending tasks)
[2024/07/31 13:49:25] [ info] [input] pausing dummy.0
leonardo@lima-default:/Users/leonardo/Work/Calyptia/fluent-bit/build$
This worked for me
Thank you so much for help @leonardo-albertovich @edsiper
Hi, I want to upload logs from fluent bit to open telemetry collector on AKS cluster
So, I have installed otel collector as deployment from here and fluentbit from here
This is my fluentbit configration for output to OTel
I have also added environment variable in fluentbit Daemonset `env:
But I am getting below error, in my fluentbit logs
Do I need to make any configuration change in Otel configmap or fluentbit to make this work? Below are the configmaps of fluentbit and oTel
Fluentbit Configmap
OTEL Configmap