search

sabre1041 / openshift-logforwarding-splunk

Demonstration of forwarding logs from OpenShift to Splunk
Apache License 2.0
30 stars 26 forks source link

fluentd pods crashing specified cert path does not exist:/var/run/ocp-collector/secrets/openshift-logforwarding-splunk/ca-bundle.crt" #2

Open walidshaari opened 4 years ago

walidshaari commented 4 years ago

Hi Andrew,

I follow your README file, installed the local splunk, however when I try to deploy the openshift-logforwarding-splunk

helm upgrade -i --namespace=openshift-logging openshift-logforwarding-splunk charts/openshift-logforwarding-splunk/ --set forwarding.splunk.token=4a8a737d-5452-426c-a6f7-106dca4e813f

fluentd pods start crashing loop and I get the error "specified cert path does not exist:/var/run/ocp-collector/secrets/openshift-logforwarding-splunk/ca-bundle.crt" so I thought maybe I am not using the right value so I specify the values.yaml file using -f option, same error, I do not do upgrade, but I uninstall and deploy 

helm upgrade -i --namespace=openshift-logging openshift-logforwarding-splunk charts/openshift-logforwarding-splunk/ -f charts/openshift-logforwarding-splunk/values.yaml --set forwarding.splunk.token=4a8a737d-5452-426c-a6f7-106dca4e813f

then I use both -f and --set

helm upgrade -i --namespace=openshift-logging openshift-logforwarding-splunk charts/openshift-logforwarding-splunk/ -f charts/openshift-logforwarding-splunk/values.yaml --set forwarding.splunk.token=4a8a737d-5452-426c-a6f7-106dca4e813f  --set forwarding.fluentd.caFile=./charts/openshift-logforwarding-splunk/files/default-openshift-logging-fluentd.crt
#oc logs fluentd-4s9xl

2020-08-11 06:54:28 +0000 [error]: config error file="/etc/fluent/fluent.conf" error_class=Fluent::ConfigError error="specified cert path does not exist:/var/run/ocp-collector/secrets/openshift-logforwarding-splunk/ca-bundle.crt"

# oc logs fluentd-mx548

2020-08-11 14:10:27 +0000 [error]: config error file="/etc/fluent/fluent.conf" error_class=Fluent::ConfigError error="specified cert path does not exist:/var/run/ocp-collector/secrets/openshift-logforwarding-splunk/ca-bundle.crt"

confession: I have not read your book yet, I promise I will do soon, any hints where I have gone wrong?

sabre1041 commented 4 years ago

Just tested it on OpenShift 4.5 using the steps described in this issue. Deployed without issues. Can you share more about your environment and any additional information that you can provide?

walidshaari commented 4 years ago

its disconnected 4.5 installation, will check again tomorrow, it is at work its when I select SSL value for Fluentd to be false.

sabre1041 commented 3 years ago

@walidshaari any update?

walidshaari commented 3 years ago

I have to enable the TLS, if I enable TLS I do not have any issues.

sabre1041 commented 3 years ago

@walidshaari Is this in fluentd as part of the integrated OCP or as part of the forwarder?

walidshaari commented 3 years ago

I believe it is the fluentd communication with each other as part of the OCP, not the forwarder

sabre1041 commented 3 years ago

@walidshaari I looked into the issue. there should be a secret called openshift-logforwarding-splunk in the openshift namespace with a key called ca-bundle.crt.

https://github.com/sabre1041/openshift-logforwarding-splunk/blob/master/charts/openshift-logforwarding-splunk/templates/log-forwarding-splunk-secret.yaml#L21

This is what the fluentd pod is trying to locate, but is failing

walidshaari commented 3 years ago

yes, I can try again next week, but no matter how I did it last time, worked