search

opensearch-project / helm-charts

:wheel_of_dharma: A community repository for Helm Charts of OpenSearch Project.
https://opensearch.org/docs/latest/opensearch/install/helm/
Apache License 2.0
159 stars 225 forks source link

how to minikube's system logs share in opensearch via fluentbit? #556

Open Ommkwn2001 opened 1 week ago

Ommkwn2001 commented 1 week ago

Please describe the end goal of this project

So i have a three helm charts opensearch, opensearch dashboard and fluentbit.

so when i try to minikube logs pass in the opensearch via fluentbit, there are give me an error in the fluentbit logs and my fluentbit version is 3.0.7 .

The error is :

[2024/06/27 05:32:52] [error] [upstream] connection opensearch-project/OpenSearch#68 to tcp://10.244.0.83:9600 timed out after 10 seconds (connection timeout) [2024/06/27 05:32:52] [error] [upstream] connection opensearch-project/OpenSearch#72 to tcp://10.244.0.83:9600 timed out after 10 seconds (connection timeout) [2024/06/27 05:32:52] [ warn] [engine] failed to flush chunk '1-1719466229.993033463.flb', retry in 87 seconds: task_id=601, input=cpu.0 > output=opensearch.0 (out_id=0) [2024/06/27 05:32:52] [ warn] [engine] failed to flush chunk '1-1719466254.994056245.flb', retry in 49 seconds: task_id=626, input=cpu.0 > output=opensearch.0 (out_id=0)

when i install fluentbit via helm i pass also this config code The code is: `config: inputs: | [INPUT] Name cpu Tag cpu

outputs: | [OUTPUT] Name opensearch Match * Host opensearch-cluster-master-headless.default.svc.cluster.local Port 9600 HTTP_User admin HTTP_Passwd TadhakDev01 Suppress_Type_Name On Trace_Error On Trace_Output On Logstash_Format On Replace_Dots On
Retry_Limit 100 Index minikubelogs `

And I edit this below code in my opensearch's value.yaml file ssl: http: false allow_unsafe_democertificates: true

Supporting References

https://opensearch.org/

Issues

So I want all logs are get from the minikube system and pass in the opensearch via fluentbit.

Related component

Clients

peternied commented 3 days ago

[Triage - attendees 1 2 3] @Ommkwn2001 Thanks for creating this issue, transferring to the helm-charts repo

@opensearch-project/admin Could you please transfer this issue to the https://github.com/opensearch-project/helm-charts repo?

Ommkwn2001 commented 2 days ago

when i try to use data prepper, so there are all logs get in opensearch dashboard through flyintbit.