open-telemetry / opentelemetry-js

OpenTelemetry JavaScript Client
https://opentelemetry.io
Apache License 2.0
2.72k stars 793 forks source link

exporter-trace-otlp-proto cannot communicate with collector version > 0.42 #2945

Closed sgracias1 closed 2 years ago

sgracias1 commented 2 years ago

Please answer these questions before submitting a bug report.

What version of OpenTelemetry are you using?

Using version 0.28 of exporter-trace-otlp-proto

What version of Node are you using?

Using version 16.0.0

Please provide the code you used to setup the OpenTelemetry SDK

tracing file

` const { HttpInstrumentation } = require('@opentelemetry/instrumentation-http'); const { NodeTracerProvider } = require("@opentelemetry/sdk-trace-node"); const { SimpleSpanProcessor } = require("@opentelemetry/sdk-trace-base"); const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-proto'); const { registerInstrumentations } = require('@opentelemetry/instrumentation'); const { SemanticResourceAttributes } = require('@opentelemetry/semantic-conventions'); const { Resource } = require('@opentelemetry/resources');

const provider = new NodeTracerProvider({ resource: new Resource({

}),
});

const collectorOptions = { url: process.env.OTC_HOST };

registerInstrumentations({ instrumentations: [new HttpInstrumentation()], });

provider.register();

const exporter = new OTLPTraceExporter(collectorOptions); provider.addSpanProcessor(new SimpleSpanProcessor(exporter)); `

Server File ` const express = require('express'), http = require('http'), pino = require('pino'), expressPino = require('express-pino-logger'), bodyParser = require('body-parser');

const logger = pino({ level: process.env.LOG_LEVEL || 'info' }); const expressLogger = expressPino({ logger }); const PORT = process.env.PORT || 6000; const app = express();

app.use(expressLogger); app.use(bodyParser.json());

app.get('/config', (req, res) => { res.statusCode = 200; res.json({status: 'OK' }); });

app.listen(PORT, () => { logger.info('Server is running on port %d', PORT); }); `

What did you do?

Running simple app instrumented for http talking to collector. Upto collector version 0.42 I see collector receive traces correctly, for versions > 0.42 no output in collector logs

If possible, provide a recipe for reproducing the error. Attaching dockerized setup basic.zip

What did you expect to see?

Traces in collector logs

What did you see instead?

Traces in collector logs for collector version <=0.42 No Traces in collector logs for collector version > 0.42

Additional context

Add any other context about the problem here. I have seen a few tickers refer to issue communicating with the collector for metrics exporter. Example https://github.com/open-telemetry/opentelemetry-js/issues/2675 and https://github.com/open-telemetry/opentelemetry-js/issues/2886 However I don't see an issue for the traces exporter.

pichlermarc commented 2 years ago

Hi! This will also be fixed by #2929. :slightly_smiling_face:

sgracias1 commented 2 years ago

@pichlermarc tyvm :)

JamieDanielson commented 2 years ago

Reviewing the files in the provided zip, it looks like the issue may be with the endpoint being used. The docker-compose.yml shows OTC_HOST=http://otel-collector:55681/v1/trace. v0.43.0 of Collector removed the path of v1/trace in PR 4720 because the path was incorrect, and changed to v1/traces; this other path v1/trace had been kept in for a year for backward compatibility but removed in v0.43.0.

If your path is changed to OTC_HOST=http://otel-collector:55681/v1/traces it seems it should work.

dyladan commented 2 years ago

I figured it would be something like that.

pichlermarc commented 2 years ago

Reviewing the files in the provided zip, it looks like the issue may be with the endpoint being used. The docker-compose.yml shows OTC_HOST=http://otel-collector:55681/v1/trace. v0.43.0 of Collector removed the path of v1/trace in PR 4720 because the path was incorrect, and changed to v1/traces; this other path v1/trace had been kept in for a year for backward compatibility but removed in v0.43.0.

If your path is changed to OTC_HOST=http://otel-collector:55681/v1/traces it seems it should work.

Ohh, that makes sense. I foolishly assumed that it was a proto issue too. I'll unlink this issue from #2886 and #2929.

dyladan commented 2 years ago

Why does it work with 50 then? did they bring back the old path?

pichlermarc commented 2 years ago

Why does it work with 50 then? did they bring back the old path?

I did not try it with this particular setup but with a basic custom example that also uses 0.28 from npm. With that it also worked for v0.43. This is why I assume that @JamieDanielson is correct. 🙂

JamieDanielson commented 2 years ago

I tested this locally to confirm this is the issue. Simplified output confirmed below:

Works: Collector 0.42.0 with OTC_HOST=http://otel-collector:55681/v1/trace

Starting otelcol... {"Version": "0.42.0", "NumCPU": 8}
...
otel-collector    | Resource SchemaURL:
otel-collector    | Resource labels:
otel-collector    |      -> service.name: STRING(send to v1/trace)
otel-collector    |      -> telemetry.sdk.language: STRING(nodejs)

Does Not Work: Collector 0.43.0 with OTC_HOST=http://otel-collector:55681/v1/trace

Starting otelcol... {"Version": "0.43.0", "NumCPU": 8}
...

Works: Collector 0.42.0 with OTC_HOST=http://otel-collector:55681/v1/traces

Starting otelcol... {"Version": "0.42.0", "NumCPU": 8}
...
otel-collector    | Resource SchemaURL:
otel-collector    | Resource labels:
otel-collector    |      -> service.name: STRING(send to v1/traces)
otel-collector    |      -> telemetry.sdk.language: STRING(nodejs)

Works: Collector 0.43.0 with OTC_HOST=http://otel-collector:55681/v1/traces

Starting otelcol... {"Version": "0.43.0", "NumCPU": 8}
...
otel-collector    | Resource SchemaURL:
otel-collector    | Resource labels:
otel-collector    |      -> service.name: STRING(send to v1/traces)
otel-collector    |      -> telemetry.sdk.language: STRING(nodejs)

I'm not sure where it was confirmed that version 50 might work; it did not work for me.

JamieDanielson commented 2 years ago

Another important note: Collector Version 0.46.0 officially removed the deprecated port 55681 with PR 4916.

So the docker-compose.yml should also change to OTC_HOST=http://otel-collector:4318/v1/traces for environment, and use "4318:4318" for ports for the collector ("4317:4317" for gRPC).

sgracias1 commented 2 years ago

Thank you! can confirm that using the new url and port I don't see this issue anymore. Closing this ticket