open-telemetry / opentelemetry-js

OpenTelemetry JavaScript Client
https://opentelemetry.io
Apache License 2.0
2.72k stars 792 forks source link

Exporter trace otlp http sending unknown field to API #3115

Closed breno-alves closed 2 years ago

breno-alves commented 2 years ago

What happened?

When trying to send data to OTLP collector receive bad request from API. I've tried with gRPC exporter and it worked just fine. May be something related to HTTP only.

Steps to Reproduce

Just try to send anything to trace

Expected Result

Expected to receive tracing on OTLP collector

Actual Result

Receive bad request from API

Additional Details

Node Version: v18.6 and v16.16

docker image: otel/opentelemetry-collector-contrib:latest

Collector conf:

receivers:
  otlp:
    protocols:
      http:
      grpc:

processors:
  batch:

exporters:
  logging:
  sentry:
    dsn: <my-self-hosted-sentry>

extensions:
  health_check:
  pprof:
  zpages:

service:
  extensions: [health_check,pprof,zpages]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging,sentry]

OpenTelemetry Setup Code

import * as opentelemetry from '@opentelemetry/sdk-node';
import { diag, DiagConsoleLogger, DiagLogLevel } from '@opentelemetry/api';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
import { SemanticResourceAttributes } from '@opentelemetry/semantic-conventions';
import { Resource } from '@opentelemetry/resources';

diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG);

const exporter = new OTLPTraceExporter({
  url: 'http://localhost:4318/v1/trace',
});

const sdk = new opentelemetry.NodeSDK({
  resource: new Resource({
    [SemanticResourceAttributes.SERVICE_NAME]: 'local-ex-node',
  }),
  instrumentations: [getNodeAutoInstrumentations()],
  traceExporter: exporter,
});

sdk
  .start()
  .then(() => {
    console.log('Tracing initialized');
  })
  .catch((error) => console.log('Error initializing tracing', error));

process.on('SIGTERM', () => {
  sdk
    .shutdown()
    .then(() => console.log('Tracing terminated'))
    .catch((error) => console.log('Error terminating tracing', error))
    .finally(() => process.exit(0));
});

package.json

"@opentelemetry/api": "^1.1.0",
"@opentelemetry/auto-instrumentations-node": "^0.31.0",
"@opentelemetry/exporter-trace-otlp-grpc": "^0.30.0",
"@opentelemetry/exporter-trace-otlp-http": "^0.30.0",
"@opentelemetry/sdk-node": "^0.30.0"

Relevant log output

{"stack":"OTLPExporterError: Bad Request
    at IncomingMessage.<anonymous> (/home/breno/.../node_modules/@opentelemetry/otlp-exporter-base/src/platform/node/util.ts:91:25)
    at /home/breno/.../node_modules/@opentelemetry/context-async-hooks/src/AbstractAsyncHooksContextManager.ts:74:49
    at AsyncLocalStorage.run (node:async_hooks:316:14)
    at AsyncLocalStorageContextManager.with (/home/breno/.../node_modules/@opentelemetry/context-async-hooks/src/AsyncLocalStorageContextManager.ts:40:36)
    at IncomingMessage.contextWrapper (/home/breno/.../node_modules/@opentelemetry/context-async-hooks/src/AbstractAsyncHooksContextManager.ts:74:26)
    at IncomingMessage.emit (node:events:539:35)
    at IncomingMessage.emit (node:domain:475:12)
    at endReadableNT (node:internal/streams/readable:1345:12)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)","message":"Bad Request","name":"OTLPExporterError","data":"{\"code\":3,\"message\":\"unknown field \\\"scopeSpans\\\" in v1.ResourceSpans\"}","code":"400"}
dyladan commented 2 years ago

There were recently some deprecated fields removed but it says the unknown field is scopeSpans which is the new field not the old field. This definitely looks like a real bug.

dyladan commented 2 years ago

Created an issue in the collector at https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/12726

breno-alves commented 2 years ago

I'm available to help if necessary :)

dyladan commented 2 years ago

I think most likely this was a change on the collector side. They may have changed the casing required in the JSON receiver or something like that. It previously allowed camelCase identifiers but they may have changed it to only allow snake_case identifiers

dyladan commented 2 years ago

@breno-alves can you capture a full JSON payload for me? You can send a span to a webhook.site endpoint to capture it

breno-alves commented 2 years ago

I get no log on OTLP collector but I can see the payload on API debug log. Is that enough? I've removed some IP data that I might not want to share.

items to be sent [
    Span {
      attributes: { 'peer.ipv4': 'ip' },
      links: [],
      events: [],
      status: { code: 0 },
      endTime: [ 1658836906, 109963617 ],
      _ended: true,
      _duration: [ 0, 15067863 ],
      name: 'dns.lookup',
      _spanContext: {
        traceId: 'd6ff8ba156bd0574d19faea3afedaa51',
        spanId: '3c45c8b03a06b3a1',
        traceFlags: 1,
        traceState: undefined
      },
      parentSpanId: undefined,
      kind: 2,
      startTime: [ 1658836906, 94895754 ],
      resource: Resource { attributes: [Object] },
      instrumentationLibrary: {
        name: '@opentelemetry/instrumentation-dns',
        version: '0.29.0',
        schemaUrl: undefined
      },
      _spanLimits: {
        attributeValueLengthLimit: Infinity,
        attributeCountLimit: 128,
        linkCountLimit: 128,
        eventCountLimit: 128
      },
      _spanProcessor: MultiSpanProcessor { _spanProcessors: [Array] },
      _attributeValueLengthLimit: Infinity
    },
    Span {
      attributes: { 'peer.ipv4': 'ip' },
      links: [],
      events: [],
      status: { code: 0 },
      endTime: [ 1658836906, 116692723 ],
      _ended: true,
      _duration: [ 0, 14617332 ],
      name: 'dns.lookup',
      _spanContext: {
        traceId: 'ee87bf3145219622cf4a81dc7457545d',
        spanId: 'ccce042ebaa5d7e0',
        traceFlags: 1,
        traceState: undefined
      },
      parentSpanId: undefined,
      kind: 2,
      startTime: [ 1658836906, 102075391 ],
      resource: Resource { attributes: [Object] },
      instrumentationLibrary: {
        name: '@opentelemetry/instrumentation-dns',
        version: '0.29.0',
        schemaUrl: undefined
      },
      _spanLimits: {
        attributeValueLengthLimit: Infinity,
        attributeCountLimit: 128,
        linkCountLimit: 128,
        eventCountLimit: 128
      },
      _spanProcessor: MultiSpanProcessor { _spanProcessors: [Array] },
      _attributeValueLengthLimit: Infinity
    },
    Span {
      attributes: { 'peer.ipv4': 'ip' },
      links: [],
      events: [],
      status: { code: 0 },
      endTime: [ 1658836906, 132841046 ],
      _ended: true,
      _duration: [ 0, 14684452 ],
      name: 'dns.lookup',
      _spanContext: {
        traceId: '48eb7e42c5c0b2416be683cd16349970',
        spanId: 'c41f82e2adf0f903',
        traceFlags: 1,
        traceState: undefined
      },
      parentSpanId: undefined,
      kind: 2,
      startTime: [ 1658836906, 118156594 ],
      resource: Resource { attributes: [Object] },
      instrumentationLibrary: {
        name: '@opentelemetry/instrumentation-dns',
        version: '0.29.0',
        schemaUrl: undefined
      },
      _spanLimits: {
        attributeValueLengthLimit: Infinity,
        attributeCountLimit: 128,
        linkCountLimit: 128,
        eventCountLimit: 128
      },
      _spanProcessor: MultiSpanProcessor { _spanProcessors: [Array] },
      _attributeValueLengthLimit: Infinity
    },
    Span {
      attributes: {
        'net.transport': 'ip_tcp',
        'net.peer.name': 'db.mongodb.net',
        'net.peer.port': 27017,
        'net.peer.ip': 'ip',
        'net.host.ip': '192.168.0.117',
        'net.host.port': 53314
      },
      links: [],
      events: [],
      status: { code: 0 },
      endTime: [ 1658836906, 193443276 ],
      _ended: true,
      _duration: [ 0, 98722722 ],
      name: 'tcp.connect',
      _spanContext: {
        traceId: '707f8f8b8d92c053b290de9ed6a295d3',
        spanId: '61e57c6ad41ee529',
        traceFlags: 1,
        traceState: undefined
      },
      parentSpanId: undefined,
      kind: 0,
      startTime: [ 1658836906, 94720554 ],
      resource: Resource { attributes: [Object] },
      instrumentationLibrary: {
        name: '@opentelemetry/instrumentation-net',
        version: '0.29.0',
        schemaUrl: undefined
      },
      _spanLimits: {
        attributeValueLengthLimit: Infinity,
        attributeCountLimit: 128,
        linkCountLimit: 128,
        eventCountLimit: 128
      },
      _spanProcessor: MultiSpanProcessor { _spanProcessors: [Array] },
      _attributeValueLengthLimit: Infinity
    },
    Span {
      attributes: {
        'net.transport': 'ip_tcp',
        'net.peer.name': 'db.mongodb.net',
        'net.peer.port': 27017,
        'net.peer.ip': 'ip',
        'net.host.ip': '192.168.0.117',
        'net.host.port': 47600
      },
      links: [],
      events: [],
      status: { code: 0 },
      endTime: [ 1658836906, 211579491 ],
      _ended: true,
      _duration: [ 0, 109588980 ],
      name: 'tcp.connect',
      _spanContext: {
        traceId: '534b82478af338355d8387088637fca2',
        spanId: 'b3cdddfac3069f95',
        traceFlags: 1,
        traceState: undefined
      },
      parentSpanId: undefined,
      kind: 0,
      startTime: [ 1658836906, 101990511 ],
      resource: Resource { attributes: [Object] },
      instrumentationLibrary: {
        name: '@opentelemetry/instrumentation-net',
        version: '0.29.0',
        schemaUrl: undefined
      },
      _spanLimits: {
        attributeValueLengthLimit: Infinity,
        attributeCountLimit: 128,
        linkCountLimit: 128,
        eventCountLimit: 128
      },
      _spanProcessor: MultiSpanProcessor { _spanProcessors: [Array] },
      _attributeValueLengthLimit: Infinity
    },
    Span {
      attributes: {
        'net.transport': 'ip_tcp',
        'net.peer.name': 'db.mongodb.net',
        'net.peer.port': 27017,
        'net.peer.ip': 'ip',
        'net.host.ip': '192.168.0.117',
        'net.host.port': 49934
      },
      links: [],
      events: [],
      status: { code: 0 },
      endTime: [ 1658836906, 228596725 ],
      _ended: true,
      _duration: [ 0, 110509301 ],
      name: 'tcp.connect',
      _spanContext: {
        traceId: '1a24ddab0c29486a5d07d496e86f4372',
        spanId: 'ecafe14c234c519e',
        traceFlags: 1,
        traceState: undefined
      },
      parentSpanId: undefined,
      kind: 0,
      startTime: [ 1658836906, 118087424 ],
      resource: Resource { attributes: [Object] },
      instrumentationLibrary: {
        name: '@opentelemetry/instrumentation-net',
        version: '0.29.0',
        schemaUrl: undefined
      },
      _spanLimits: {
        attributeValueLengthLimit: Infinity,
        attributeCountLimit: 128,
        linkCountLimit: 128,
        eventCountLimit: 128
      },
      _spanProcessor: MultiSpanProcessor { _spanProcessors: [Array] },
      _attributeValueLengthLimit: Infinity
    },
    Span {
      attributes: {
        'tls.protocol': 'TLSv1.3',
        'tls.authorized': 'true',
        'tls.cipher.name': 'TLS_AES_128_GCM_SHA256',
        'tls.cipher.version': 'TLSv1.3',
        'tls.certificate.fingerprint': '4D:21:BA:89:9D:8A:B6:DC:5E:A6:99:C7:88:8F:21:67:38:E6:AB:94',
        'tls.certificate.serialNumber': '04825D461F43B1D5753EBF702D3989D04E34',
        'tls.certificate.validFrom': 'Jun  6 10:17:07 2022 GMT',
        'tls.certificate.validTo': 'Sep  4 10:17:06 2022 GMT',
        'tls.alpnProtocol': ''
      },
      links: [],
      events: [],
      status: { code: 0 },
      endTime: [ 1658836906, 288219054 ],
      _ended: true,
      _duration: [ 0, 193551160 ],
      name: 'tls.connect',
      _spanContext: {
        traceId: '4055c85499d2487d75ba5d58213fdf9a',
        spanId: '8d7be56ce70f9630',
        traceFlags: 1,
        traceState: undefined
      },
      parentSpanId: undefined,
      kind: 0,
      startTime: [ 1658836906, 94667894 ],
      resource: Resource { attributes: [Object] },
      instrumentationLibrary: {
        name: '@opentelemetry/instrumentation-net',
        version: '0.29.0',
        schemaUrl: undefined
      },
      _spanLimits: {
        attributeValueLengthLimit: Infinity,
        attributeCountLimit: 128,
        linkCountLimit: 128,
        eventCountLimit: 128
      },
      _spanProcessor: MultiSpanProcessor { _spanProcessors: [Array] },
      _attributeValueLengthLimit: Infinity
    },
    Span {
      attributes: {
        'tls.protocol': 'TLSv1.3',
        'tls.authorized': 'true',
        'tls.cipher.name': 'TLS_AES_128_GCM_SHA256',
        'tls.cipher.version': 'TLSv1.3',
        'tls.certificate.fingerprint': '4D:21:BA:89:9D:8A:B6:DC:5E:A6:99:C7:88:8F:21:67:38:E6:AB:94',
        'tls.certificate.serialNumber': '04825D461F43B1D5753EBF702D3989D04E34',
        'tls.certificate.validFrom': 'Jun  6 10:17:07 2022 GMT',
        'tls.certificate.validTo': 'Sep  4 10:17:06 2022 GMT',
        'tls.alpnProtocol': ''
      },
      links: [],
      events: [],
      status: { code: 0 },
      endTime: [ 1658836906, 318110549 ],
      _ended: true,
      _duration: [ 0, 216148918 ],
      name: 'tls.connect',
      _spanContext: {
        traceId: '942d27b3f1848257f4e19922ba7ddb7e',
        spanId: '03cf14fb4e4f49c4',
        traceFlags: 1,
        traceState: undefined
      },
      parentSpanId: undefined,
      kind: 0,
      startTime: [ 1658836906, 101961631 ],
      resource: Resource { attributes: [Object] },
      instrumentationLibrary: {
        name: '@opentelemetry/instrumentation-net',
        version: '0.29.0',
        schemaUrl: undefined
      },
      _spanLimits: {
        attributeValueLengthLimit: Infinity,
        attributeCountLimit: 128,
        linkCountLimit: 128,
        eventCountLimit: 128
      },
      _spanProcessor: MultiSpanProcessor { _spanProcessors: [Array] },
      _attributeValueLengthLimit: Infinity
    },
    Span {
      attributes: {
        'tls.protocol': 'TLSv1.3',
        'tls.authorized': 'true',
        'tls.cipher.name': 'TLS_AES_128_GCM_SHA256',
        'tls.cipher.version': 'TLSv1.3',
        'tls.certificate.fingerprint': '4D:21:BA:89:9D:8A:B6:DC:5E:A6:99:C7:88:8F:21:67:38:E6:AB:94',
        'tls.certificate.serialNumber': '04825D461F43B1D5753EBF702D3989D04E34',
        'tls.certificate.validFrom': 'Jun  6 10:17:07 2022 GMT',
        'tls.certificate.validTo': 'Sep  4 10:17:06 2022 GMT',
        'tls.alpnProtocol': ''
      },
      links: [],
      events: [],
      status: { code: 0 },
      endTime: [ 1658836906, 339300877 ],
      _ended: true,
      _duration: [ 0, 221233113 ],
      name: 'tls.connect',
      _spanContext: {
        traceId: '362ce4e428dd4a774a56c8a90090aec4',
        spanId: '613f59b42561a860',
        traceFlags: 1,
        traceState: undefined
      },
      parentSpanId: undefined,
      kind: 0,
      startTime: [ 1658836906, 118067764 ],
      resource: Resource { attributes: [Object] },
      instrumentationLibrary: {
        name: '@opentelemetry/instrumentation-net',
        version: '0.29.0',
        schemaUrl: undefined
      },
      _spanLimits: {
        attributeValueLengthLimit: Infinity,
        attributeCountLimit: 128,
        linkCountLimit: 128,
        eventCountLimit: 128
      },
      _spanProcessor: MultiSpanProcessor { _spanProcessors: [Array] },
      _attributeValueLengthLimit: Infinity
    }
  ]
dyladan commented 2 years ago

It would be better to capture the actual JSON. One way to do that would be to change the URL of your sample from the collector endpoint to an endpoint like webhook.site

breno-alves commented 2 years ago

Give me just a minute, removing ip data

dyladan commented 2 years ago

Can you also check if a previous version of the collector works?

breno-alves commented 2 years ago

got this one from a http server

var http = require('http');

console.log("server initialized");

var server = http.createServer(function (req, response) {
    let chunks = [];

    req.on('data', (chunk) => {
        chunks.push(Buffer.from(chunk, 'binary'));
    });

    req.on('end', () => {
        let binary = Buffer.concat(chunks);
        console.log(binary.toString())
   });

}).listen(4444);
{
    "resourceSpans": [
        {
            "resource": {
                "attributes": [
                    {
                        "key": "service.name",
                        "value": {
                            "stringValue": "local-ex-node"
                        }
                    },
                    {
                        "key": "telemetry.sdk.language",
                        "value": {
                            "stringValue": "nodejs"
                        }
                    },
                    {
                        "key": "telemetry.sdk.name",
                        "value": {
                            "stringValue": "opentelemetry"
                        }
                    },
                    {
                        "key": "telemetry.sdk.version",
                        "value": {
                            "stringValue": "1.4.0"
                        }
                    },
                    {
                        "key": "process.pid",
                        "value": {
                            "intValue": 519509
                        }
                    },
                    {
                        "key": "process.executable.name",
                        "value": {
                            "stringValue": "/home/breno/.nvm/versions/node/v16.16.0/bin/node"
                        }
                    },
                    {
                        "key": "process.command",
                        "value": {
                            "stringValue": "/home/breno.../dist/src/main"
                        }
                    },
                    {
                        "key": "process.command_line",
                        "value": {
                            "stringValue": "/home/breno/.nvm/versions/node/v16.16.0/bin/node /home/breno.../dist/src/main"
                        }
                    },
                    {
                        "key": "process.runtime.version",
                        "value": {
                            "stringValue": "16.16.0"
                        }
                    },
                    {
                        "key": "process.runtime.name",
                        "value": {
                            "stringValue": "nodejs"
                        }
                    },
                    {
                        "key": "process.runtime.description",
                        "value": {
                            "stringValue": "Node.js"
                        }
                    }
                ],
                "droppedAttributesCount": 0
            },
            "scopeSpans": [
                {
                    "scope": {
                        "name": "@opentelemetry/instrumentation-express",
                        "version": "0.30.0"
                    },
                    "spans": [
                        {
                            "traceId": "43a6e6ed7f18c04620141e39d395db19",
                            "spanId": "bea17e9a9ea5d9e7",
                            "parentSpanId": "2ad2b0e21f837b00",
                            "name": "middleware - query",
                            "kind": 1,
                            "startTimeUnixNano": 1658842452778355500,
                            "endTimeUnixNano": 1658842452778403000,
                            "attributes": [
                                {
                                    "key": "http.route",
                                    "value": {
                                        "stringValue": "/"
                                    }
                                },
                                {
                                    "key": "express.name",
                                    "value": {
                                        "stringValue": "query"
                                    }
                                },
                                {
                                    "key": "express.type",
                                    "value": {
                                        "stringValue": "middleware"
                                    }
                                }
                            ],
                            "droppedAttributesCount": 0,
                            "events": [],
                            "droppedEventsCount": 0,
                            "status": {
                                "code": 0
                            },
                            "links": [],
                            "droppedLinksCount": 0
                        },
                        {
                            "traceId": "43a6e6ed7f18c04620141e39d395db19",
                            "spanId": "4fd112dfbad74ad7",
                            "parentSpanId": "2ad2b0e21f837b00",
                            "name": "middleware - expressInit",
                            "kind": 1,
                            "startTimeUnixNano": 1658842452778466800,
                            "endTimeUnixNano": 1658842452778506800,
                            "attributes": [
                                {
                                    "key": "http.route",
                                    "value": {
                                        "stringValue": "/"
                                    }
                                },
                                {
                                    "key": "express.name",
                                    "value": {
                                        "stringValue": "expressInit"
                                    }
                                },
                                {
                                    "key": "express.type",
                                    "value": {
                                        "stringValue": "middleware"
                                    }
                                }
                            ],
                            "droppedAttributesCount": 0,
                            "events": [],
                            "droppedEventsCount": 0,
                            "status": {
                                "code": 0
                            },
                            "links": [],
                            "droppedLinksCount": 0
                        },
                        {
                            "traceId": "43a6e6ed7f18c04620141e39d395db19",
                            "spanId": "c37dee6cb3fd9794",
                            "parentSpanId": "2ad2b0e21f837b00",
                            "name": "middleware - corsMiddleware",
                            "kind": 1,
                            "startTimeUnixNano": 1658842452778609700,
                            "endTimeUnixNano": 1658842452778647000,
                            "attributes": [
                                {
                                    "key": "http.route",
                                    "value": {
                                        "stringValue": "/"
                                    }
                                },
                                {
                                    "key": "express.name",
                                    "value": {
                                        "stringValue": "corsMiddleware"
                                    }
                                },
                                {
                                    "key": "express.type",
                                    "value": {
                                        "stringValue": "middleware"
                                    }
                                }
                            ],
                            "droppedAttributesCount": 0,
                            "events": [],
                            "droppedEventsCount": 0,
                            "status": {
                                "code": 0
                            },
                            "links": [],
                            "droppedLinksCount": 0
                        },
                        {
                            "traceId": "43a6e6ed7f18c04620141e39d395db19",
                            "spanId": "d2d5d54491590fe0",
                            "parentSpanId": "2ad2b0e21f837b00",
                            "name": "middleware - jsonParser",
                            "kind": 1,
                            "startTimeUnixNano": 1658842452778681600,
                            "endTimeUnixNano": 1658842452778708500,
                            "attributes": [
                                {
                                    "key": "http.route",
                                    "value": {
                                        "stringValue": "/"
                                    }
                                },
                                {
                                    "key": "express.name",
                                    "value": {
                                        "stringValue": "jsonParser"
                                    }
                                },
                                {
                                    "key": "express.type",
                                    "value": {
                                        "stringValue": "middleware"
                                    }
                                }
                            ],
                            "droppedAttributesCount": 0,
                            "events": [],
                            "droppedEventsCount": 0,
                            "status": {
                                "code": 0
                            },
                            "links": [],
                            "droppedLinksCount": 0
                        },
                        {
                            "traceId": "43a6e6ed7f18c04620141e39d395db19",
                            "spanId": "72d6e26296c732ea",
                            "parentSpanId": "2ad2b0e21f837b00",
                            "name": "middleware - urlencodedParser",
                            "kind": 1,
                            "startTimeUnixNano": 1658842452778742800,
                            "endTimeUnixNano": 1658842452778764000,
                            "attributes": [
                                {
                                    "key": "http.route",
                                    "value": {
                                        "stringValue": "/"
                                    }
                                },
                                {
                                    "key": "express.name",
                                    "value": {
                                        "stringValue": "urlencodedParser"
                                    }
                                },
                                {
                                    "key": "express.type",
                                    "value": {
                                        "stringValue": "middleware"
                                    }
                                }
                            ],
                            "droppedAttributesCount": 0,
                            "events": [],
                            "droppedEventsCount": 0,
                            "status": {
                                "code": 0
                            },
                            "links": [],
                            "droppedLinksCount": 0
                        },
                        {
                            "traceId": "43a6e6ed7f18c04620141e39d395db19",
                            "spanId": "fb115a3ef588e330",
                            "parentSpanId": "2ad2b0e21f837b00",
                            "name": "request handler - /v1/referral-marketing/generated-magazines",
                            "kind": 1,
                            "startTimeUnixNano": 1658842452778880000,
                            "endTimeUnixNano": 1658842452778888000,
                            "attributes": [
                                {
                                    "key": "http.route",
                                    "value": {
                                        "stringValue": "/v1/referral-marketing/generated-magazines"
                                    }
                                },
                                {
                                    "key": "express.name",
                                    "value": {
                                        "stringValue": "/v1/referral-marketing/generated-magazines"
                                    }
                                },
                                {
                                    "key": "express.type",
                                    "value": {
                                        "stringValue": "request_handler"
                                    }
                                }
                            ],
                            "droppedAttributesCount": 0,
                            "events": [],
                            "droppedEventsCount": 0,
                            "status": {
                                "code": 0
                            },
                            "links": [],
                            "droppedLinksCount": 0
                        }
                    ]
                },
                {
                    "scope": {
                        "name": "@opentelemetry/instrumentation-nestjs-core",
                        "version": "0.30.0"
                    },
                    "spans": [
                        {
                            "traceId": "43a6e6ed7f18c04620141e39d395db19",
                            "spanId": "bb54c0f31f913727",
                            "parentSpanId": "2ad2b0e21f837b00",
                            "name": "GeneratedMagazinesController.findAll",
                            "kind": 1,
                            "startTimeUnixNano": 1658842452779250200,
                            "endTimeUnixNano": 1658842452779736600,
                            "attributes": [
                                {
                                    "key": "component",
                                    "value": {
                                        "stringValue": "@nestjs/core"
                                    }
                                },
                                {
                                    "key": "nestjs.version",
                                    "value": {
                                        "stringValue": "8.4.0"
                                    }
                                },
                                {
                                    "key": "nestjs.type",
                                    "value": {
                                        "stringValue": "request_context"
                                    }
                                },
                                {
                                    "key": "http.method",
                                    "value": {
                                        "stringValue": "GET"
                                    }
                                },
                                {
                                    "key": "http.url",
                                    "value": {
                                        "stringValue": "/v1/referral-marketing/generated-magazines/"
                                    }
                                },
                                {
                                    "key": "http.route",
                                    "value": {
                                        "stringValue": "/v1/referral-marketing/generated-magazines"
                                    }
                                },
                                {
                                    "key": "nestjs.controller",
                                    "value": {
                                        "stringValue": "GeneratedMagazinesController"
                                    }
                                },
                                {
                                    "key": "nestjs.callback",
                                    "value": {
                                        "stringValue": "findAll"
                                    }
                                }
                            ],
                            "droppedAttributesCount": 0,
                            "events": [
                                {
                                    "attributes": [
                                        {
                                            "key": "exception.type",
                                            "value": {
                                                "stringValue": "UnauthorizedException"
                                            }
                                        },
                                        {
                                            "key": "exception.message",
                                            "value": {
                                                "stringValue": "Unauthorized"
                                            }
                                        },
                                        {
                                            "key": "exception.stacktrace",
                                            "value": {
                                                "stringValue": "UnauthorizedException: Unauthorized\n    at JwtGuard.handleRequest (/home/breno.../node_modules/@nestjs/passport/dist/auth.guard.js:68:30)\n    at /home/breno.../node_modules/@nestjs/passport/dist/auth.guard.js:49:128\n    at /home/breno.../node_modules/@nestjs/passport/dist/auth.guard.js:91:24\n    at allFailed (/home/breno.../node_modules/passport/lib/middleware/authenticate.js:110:18)\n    at attempt (/home/breno.../node_modules/passport/lib/middleware/authenticate.js:183:28)\n    at JwtStrategy.strategy.fail (/home/breno.../node_modules/passport/lib/middleware/authenticate.js:305:9)\n    at /home/breno.../node_modules/passport-jwt/lib/strategy.js:106:33\n    at /home/breno.../node_modules/jsonwebtoken/verify.js:152:16\n    at getSecret (/home/breno.../node_modules/jsonwebtoken/verify.js:90:14)\n    at Object.module.exports [as verify] (/home/breno.../node_modules/jsonwebtoken/verify.js:94:10)"
                                            }
                                        }
                                    ],
                                    "name": "exception",
                                    "timeUnixNano": 1658842452779562500,
                                    "droppedAttributesCount": 0
                                }
                            ],
                            "droppedEventsCount": 0,
                            "status": {
                                "code": 2,
                                "message": "Unauthorized"
                            },
                            "links": [],
                            "droppedLinksCount": 0
                        }
                    ]
                },
                {
                    "scope": {
                        "name": "@opentelemetry/instrumentation-http",
                        "version": "0.29.2"
                    },
                    "spans": [
                        {
                            "traceId": "43a6e6ed7f18c04620141e39d395db19",
                            "spanId": "2ad2b0e21f837b00",
                            "name": "GET /v1/referral-marketing/generated-magazines",
                            "kind": 2,
                            "startTimeUnixNano": 1658842452778180900,
                            "endTimeUnixNano": 1658842452780349000,
                            "attributes": [
                                {
                                    "key": "http.url",
                                    "value": {
                                        "stringValue": "http://localhost:4000/v1/referral-marketing/generated-magazines/"
                                    }
                                },
                                {
                                    "key": "http.host",
                                    "value": {
                                        "stringValue": "localhost:4000"
                                    }
                                },
                                {
                                    "key": "net.host.name",
                                    "value": {
                                        "stringValue": "localhost"
                                    }
                                },
                                {
                                    "key": "http.method",
                                    "value": {
                                        "stringValue": "GET"
                                    }
                                },
                                {
                                    "key": "http.target",
                                    "value": {
                                        "stringValue": "/v1/referral-marketing/generated-magazines/"
                                    }
                                },
                                {
                                    "key": "http.user_agent",
                                    "value": {
                                        "stringValue": "PostmanRuntime/7.29.2"
                                    }
                                },
                                {
                                    "key": "http.flavor",
                                    "value": {
                                        "stringValue": "1.1"
                                    }
                                },
                                {
                                    "key": "net.transport",
                                    "value": {
                                        "stringValue": "ip_tcp"
                                    }
                                },
                                {
                                    "key": "net.host.ip",
                                    "value": {
                                        "stringValue": "::1"
                                    }
                                },
                                {
                                    "key": "net.host.port",
                                    "value": {
                                        "intValue": 4000
                                    }
                                },
                                {
                                    "key": "net.peer.ip",
                                    "value": {
                                        "stringValue": "::1"
                                    }
                                },
                                {
                                    "key": "net.peer.port",
                                    "value": {
                                        "intValue": 57446
                                    }
                                },
                                {
                                    "key": "http.status_code",
                                    "value": {
                                        "intValue": 401
                                    }
                                },
                                {
                                    "key": "http.status_text",
                                    "value": {
                                        "stringValue": "UNAUTHORIZED"
                                    }
                                },
                                {
                                    "key": "http.route",
                                    "value": {
                                        "stringValue": "/v1/referral-marketing/generated-magazines"
                                    }
                                }
                            ],
                            "droppedAttributesCount": 0,
                            "events": [],
                            "droppedEventsCount": 0,
                            "status": {
                                "code": 0
                            },
                            "links": [],
                            "droppedLinksCount": 0
                        }
                    ]
                }
            ]
        }
    ]
}
breno-alves commented 2 years ago

Can you also check if a previous version of the collector works?

checking...

breno-alves commented 2 years ago

otel/opentelemetry-collector-contrib:0.53.0

working just fine, I'll test on which version it is breaking...

@dyladan

breno-alves commented 2 years ago

0.54.0 working fine 0.55.0 working fine 0.56.0 working fine...

I just realized that :latest isn't the lastest version... there is a image tagged as latest from 7 months

https://hub.docker.com/layers/opentelemetry-collector-contrib/otel/opentelemetry-collector-contrib/latest/images/sha256-d34519458388e55a3fce38a33e6bc424267c1f432927c09e932ba45f7575bd84?context=explore

maybe this is the problem?

dyladan commented 2 years ago

Yes that is exactly the problem. scopeSpans was introduced to replace instrumentationLibrarySpans several months ago.

dyladan commented 2 years ago

Closed since this was just a version mixup

dyladan commented 2 years ago

https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/12729

mbrevda commented 2 years ago

Seeing this with "@opentelemetry/exporter-trace-otlp-http": "^0.31.0", not using docker for exporting. Is there a better version I should be using?

anthony-telljohann commented 2 years ago

I'm experiencing the same issue. I narrowed it down to a change in @opentelemetry/exporter-trace-otlp-http@0.29.0. Could we please open and resolve this issue?

guyguy333 commented 2 years ago

If you're using Tempo, you should check this issue: https://github.com/grafana/tempo/issues/1746 In that case, consider using another exporter like Zipkin waiting a fix