Closed argais closed 1 year ago
Thanks for creating this, it seems totally reasonable.
i have the same problem too
This looks like https://github.com/uken/fluent-plugin-elasticsearch/issues/935
Have you tried to configure max_retry_putting_template
to a higher value? This solution seems a bit fragile but might work.
I'm deploying Logging Operator and my output (Loki) at the same time and have no issues as I use ArgoCD Waves to order the deployments. It might fit your need too.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!
Giving this another thought: trying to connect and write to an output during dry-run
doesn't seem to be right to me, so I wouldn't say we could do anything about this other than probably make the configcheck pod optionally retryable, but I'm not sure it would worth the effort.
Closing this for now, but feel free to reopen if you think we should discuss this further.
Describe the bug: On a freshly deployed cluster, ArgoCD deploys the logging operator with a clusterflow and clusteroutput that pushes logs to elasticsearch.
Since both are deployed at the same time, elasticsearch is not readily available and the fluentd configcheck pod fails, and stays stuck forever until deleted.
Deleting it manually is less than desirable as we wanted these clusters to come up without manual intervention.
Expected behaviour: Configcheck to fail and retry until it eventually succeeds
Steps to reproduce the bug: Deploy the logging operator using helm, latest version.
Cluster flow
Cluster output
Fluentd config check pod will fail, with logs as
Additional context: If you delete the config check pod it will retry and fluentd/bit pods will come up as expected, but manual intervention is not really desirable.
Environment details:
/kind bug