search

rancher / dashboard

The Rancher UI
https://rancher.com
Apache License 2.0
441 stars 244 forks source link

Logging 2.5 (new) - logging output with Kafka -> error in configuration #5158

Open bramamad opened 3 years ago

bramamad commented 3 years ago

What kind of request is this (question/bug/enhancement/feature request):

Steps to reproduce (least amount of steps as possible):

Result:

Other details that may be helpful:

ssl_client_cert -----BEGIN CERTIFICATE----- MIIGuDCCBaCgAwIBAgIUGhRjYZfN/5vrp6g81IWZiBFEnaowDQYJKoZIhvcNAQEL .............. 9IK9neCAOVxetolNOHhJ9Nk4ikimbMVK05ccBminuA58YKy7GdW3JkW2gl8= -----END CERTIFICATE-----

ssl_client_cert_key -----BEGIN PRIVATE KEY----- MIIEvgIBADAN...................... BdHSVLoJvXTOmHNLQfqQuNuZpLzlzSS44oRg26lr8AOWVaCfRPZ1Qw1UiqyIB6OP nh4qO1W8QOa10RnTBGyldJOu -----END PRIVATE KEY-----

ssl_client_cert /fluentd/etc/config/ssl/cluster_c-9d8lb_client-cert.pem ssl_client_cert_key /fluentd/etc/config/ssl/cluster_c-9d8lb_client-key.pem container (1).log

Environment information

bramamad commented 3 years ago

Will someone have a look at this error ? Or what can I do for solving this problem ?

bsauvajon commented 3 years ago

Hello, I'm having the same problem, did you find a solution ?

bramamad commented 3 years ago

Hi, no I never got a solution for this fluentd problem. - Now I changed it and we use filebeat with kafka output as log collector. That works and is a lot more easier. Unfortunately, it is a site specific solution.

bsauvajon commented 3 years ago

Hello, I finally found the solution in banzaicloud documentation, edit the output yaml configuration and replace the keys "valueFrom" by "mountFrom". It will automaticcaly mount the secrets in files, and use these files in the fluentd configuration. It would be great if rancher could generate the configuration with the right keys !

stale[bot] commented 3 years ago

This repository uses a bot to automatically label issues which have not had any activity (commit/comment/label) for 60 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the bot can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the bot will automatically close the issue in 14 days. Thank you for your contributions.

bdekany commented 2 years ago

Similar issue with Splunk https://github.com/rancher/rancher/issues/31810

paynejacob commented 2 years ago

This is a bug in the upstream chart https://github.com/banzaicloud/logging-operator/issues/833 I do not know of any workarounds at this time.

snasovich commented 2 years ago

Setting as “Need Info” as full-in for Blocked status as we’re blocked by upstream issue.

belgaied2 commented 2 years ago

There is an underlying Rancher UI bug here, since, the form for ClusterOutput, let's say for elasticsearch, will offer the user to choose a secret (for example as a ca_file) image

and would generate in the YAML valueFrom instead of the expected mountFrom: image

deniseschannon commented 2 years ago

@belgaied2 Does that mean if we did the yaml correctly, then this use case would work?

belgaied2 commented 2 years ago

@deniseschannon That's what I tested, the ClusterOutput would work correctly (at least in my tests with Elasticsearch). As the Logging Operator is supposed to take mountFrom instead of valueFrom, the bug referenced by @paynejacob is actually one that can cause misconfiguration because valueFrom is accepted for SSL certificates where it should not!

As mentioned above by @bsauvajon _

Hello, I finally found the solution in banzaicloud documentation, edit the output yaml configuration and replace the keys "valueFrom" by "mountFrom". It will automaticcaly mount the secrets in files, and use these files in the fluentd configuration. It would be great if rancher could generate the configuration with the right keys !

So, basically, though Rancher is capable through the UI abstraction to avoid the possible misconfiguration by generating the right mountFrom, it does cause the misconfiguration by generating valueFrom.

So, yes, short answer is : generating the yaml correctly would most certainly solve the issue/use case.

belgaied2 commented 2 years ago

BTW @deniseschannon the problem at hand is better described in this issue : rancher/dashboard#5157

deniseschannon commented 2 years ago

Thanks for your detail @belgaied2!

Since these look like UI related issues, I've transferred them into the dashboard repo, but in general, any and all UI issues should be filed in rancher/dashboard