evanderkoogh / otel-cf-workers

An OpenTelemetry compatible library for instrumenting and exporting traces for Cloudflare Workers
BSD 3-Clause "New" or "Revised" License
239 stars 50 forks source link

cannot get this working with New Relic #39

Closed willie closed 1 year ago

willie commented 1 year ago

I've been unable to get this working with New Relic. This is my config.

const config: ResolveConfigFn = (env: Env, _trigger) => {
    return {
        exporter: {
            // new relic
            url: 'https://otlp.nr-data.net:4318/v1/traces',
            headers: { 'api-key': env.NR_API_KEY },
        },
        service: { name: env.NR_SERVICE_NAME },
    }
}

and this is the error:

[mf:inf] Updated and ready on http://127.0.0.1:8787/
WARNING: known issue with `fetch()` requests to custom HTTPS ports in published Workers:
 - https://otlp.nr-data.net:4318/ - the custom port will be ignored when the Worker is published using the `wrangler deploy` command.

[mf:inf] GET / 200 OK (20ms)
Error sending spans to exporter: OTLPExporterError: Exception during export: Error: Network connection lost.
    at new OTLPExporterError2 (worker.js:2746:24)
    at worker.js:2845:15 {
  name: OTLPExporterError,
  data: Error: Network connection lost.,
  code: undefined,
  stack: OTLPExporterError: Exception during export: Error:…ror2 (worker.js:2746:24)
    at worker.js:2845:15,
  message: Exception during export: Error: Network connection lost.
}

I have the .dev.vars file with the secret set. I also tried set the value directly in code. Same error. I get the same error with no api-key header or complete gibberish as an api-key header value.

This configuration, which is similar, works fine with Jaeger and SigNoz running on localhost.

const config: ResolveConfigFn = (env: Env, _trigger) => {
    return {
        exporter: {
            url: 'http://localhost:4318/v1/traces',
        },
        service: { name: env.NR_SERVICE_NAME },
    }
}

I've updated the issue here from our discussion from discord. https://discord.com/channels/595317990191398933/1093094892080738394/1139831369107767406

willie commented 1 year ago

I’ve spent some time trying out other OpenTelemetry vendor integrations and I've confirmed:

HTTP endpoints with non-standard HTTP port numbers work. http://localhost:4318/v1/traces HTTPS endpoints on standard HTTPS (443) ports work: https://otlp.uptrace.dev/v1/traces HTTPS endpoints on custom ports do not work: https://otlp.nr-data.net:4318/v1/traces

However, it's not your bug. This is a Cloudflare Worker runtime issue: ...fetch ignoring custom (HTTPS) ports is considered behaviour as designed...

In order to instrument Cloudflare Workers at this time, New Relic would need an OpenTelemetry endpoint that is HTTPS on standard HTTPS ports, or HTTP on any port.

Not sure where you want to leave this issue as a result.

willie commented 1 year ago

New Relic doesn't work on the standard HTTPS port either. I've been working with their tech support on this. It seems peculiar to New Relic only.

willie commented 1 year ago

Huge apology here. It turns out that the issue, this entire time, was NextDNS having otlp.nr-data.net on some blocking list and nothing to do with this library or New Relic. I don't know if you can log the hostname and IP address on those kinds of connection errors, but it might be helpful to others in the future. Again my apologies.

You have to use https://otlp.nr-data.net/v1/traces as the endpoint, though. Custom port numbers do not work.

I'M NOT SAYING IT'S ONS