open-telemetry / opentelemetry-helm-charts

OpenTelemetry Helm Charts
https://opentelemetry.io
Apache License 2.0
385 stars 464 forks source link

We are facing Error scraping metrics tls: failed to verify certificate: x509: certificate signed by unknown authority #1247

Open shbasha-clgx opened 2 months ago

shbasha-clgx commented 2 months ago

Hi Team, We are trying to install otel collector using helm and trying to get the kubernetesAttributes: enabled: true kubeletMetrics: enabled: true hostMetrics: enabled: true logsCollection: enabled: true includeCollectorLogs:

configuration added in values.yaml how ever we are facing below error , attached file for your reference.

Kindly look into this and please assist

error scraperhelper/scrapercontroller.go:197 Error scraping metrics {"kind": "receiver", "name": "kubeletstats", "data_type": "metrics", "error": "Get \" [https://xxxxxx:xxxx/stats/summary\":] tls: failed to verify certificate: x509: certificate signed by unknown authority", "scraper": "kubeletstats"}

we do not have self signed cert and need assistance on how to bypass this.

shbasha-clgx commented 2 months ago

@dmitryax Hi , can someone assist on this issue please

shbasha-clgx commented 1 month ago

@dmitryax Hi did anyone got a chance to go through the issue, we still see the same issue

povilasv commented 1 month ago

you can try enabling preset and also providing receiver config:

receivers:
  kubeletstats:
    insecure_skip_verify: true

Check this discussion around tls issues - https://github.com/open-telemetry/opentelemetry-helm-charts/pull/1034

shbasha-clgx commented 1 month ago

you can try enabling preset and also providing receiver config:

receivers:
  kubeletstats:
    insecure_skip_verify: true

Check this discussion around tls issues - #1034

@povilasv Thank you for your response after adding receivers:

kubeletstats: insecure_skip_verify: true

we not seeing the tls error now but we are not able to get the cluster metrics in the otel collector logs although as exporter we put debug , below the values.yaml file we are using.

mode: daemonset

presets: kubernetesAttributes: enabled: true kubeletMetrics: enabled: true hostMetrics: enabled: true logsCollection: enabled: true includeCollectorLogs: true

config: exporters: debug: {} # Enable OTLP HTTP exporter service: pipelines: logs: exporters:

nodeSelector: kubernetes.io/hostname: xxxxxx tolerations:

shbasha-clgx commented 1 month ago

@povilasv

Thank you for your response after adding receivers:

kubeletstats: insecure_skip_verify: true

We are not seeing the tls error now but we are not able to get the cluster metrics in the otel collector logs although as exporter we put debug , below the values.yaml file we are using.

mode: daemonset

presets: kubernetesAttributes: enabled: true kubeletMetrics: enabled: true hostMetrics: enabled: true logsCollection: enabled: true includeCollectorLogs: true

config: exporters: debug: {} # Enable OTLP HTTP exporter service: pipelines: logs: exporters:

nodeSelector: kubernetes.io/hostname: xxxxxx tolerations:

shbasha-clgx commented 1 month ago

@povilasv Hi , did you get a chance to look into this issue , can you please assist.

povilasv commented 1 month ago

Try enabling debug logs and seeing what is happening https://opentelemetry.io/docs/collector/internal-telemetry/#configure-internal-logs

bixu commented 3 weeks ago

below the values.yaml file we are using.

@shbasha-clgx, it'll help the community debug your issue if you paste formatted YAML.

You can do that by surrounding your YAML blocks with triple backticks, or selecting the YAML you've pasted and then clicking on the <> in the comment editor.