Datadog tracing not working

imatefx commented 3 years ago

Prerequisites

Please answer the following questions for yourself before submitting an issue.

[x] I am running the latest version
[x] I checked the documentation and found no answer
[x] I checked to make sure that this issue has not already been filed
[x] I'm reporting the issue to the correct repository

Current Behavior

 TypeError: Cannot read property '64039' of undefined
    at DatadogTraceExporter.spanStarted (/home/stalin/dev/backend-service/node_modules/moleculer/src/tracing/exporters/datadog.js:158:38)
    at /home/stalin/dev/backend-service/node_modules/moleculer/src/tracing/tracer.js:219:55
    at Array.forEach (<anonymous>)
    at Tracer.invokeExporter (/home/stalin/dev/backend-service/node_modules/moleculer/src/tracing/tracer.js:219:18)
    at Tracer.spanStarted (/home/stalin/dev/backend-service/node_modules/moleculer/src/tracing/tracer.js:300:9)
    at Span.start (/home/stalin/dev/backend-service/node_modules/moleculer/src/tracing/span.js:87:15)
    at Tracer.startSpan (/home/stalin/dev/backend-service/node_modules/moleculer/src/tracing/tracer.js:205:8)
    at Context.startSpan (/home/stalin/dev/backend-service/node_modules/moleculer/src/context.js:412:30)
    at ServiceBroker.tracingLocalActionMiddleware (/home/stalin/dev/backend-service/node_modules/moleculer/src/middlewares/tracing.js:84:22)
    at metricsMiddleware (/home/stalin/dev/backend-service/node_modules/moleculer/src/middlewares/metrics.js:27:11)
    at Object.actions.<computed> [as rest] (/home/stalin/dev/backend-service/node_modules/moleculer/src/service.js:157:13)
    at Service.httpHandler (/home/stalin/dev/backend-service/node_modules/moleculer-web/src/index.js:343:39)
    at Server.emit (events.js:315:20)
    at /home/stalin/dev/backend-service/node_modules/dd-trace/packages/datadog-plugin-http/src/server.js:13:23
    at /home/stalin/dev/backend-service/node_modules/dd-trace/packages/dd-trace/src/plugins/util/web.js:74:60
    at Scope._activate (/home/stalin/dev/backend-service/node_modules/dd-trace/packages/dd-trace/src/scope/async_resource.js:47:14)

Expected Behavior

Tracing spans should be pushed to datadog

Failure Information

Steps to Reproduce

Please provide detailed steps for reproducing the issue.

Install ddtrace lib
Add tracing config to moleculer.config.js

{
    type: "Datadog",
    options: {
        // Datadog Agent URL
        agentUrl:
            process.env.DD_AGENT_URL,
        // Environment variable
        env: process.env.DD_ENVIRONMENT || null,
        // Sampling priority. More info: https://docs.datadoghq.com/tracing/guide/trace_sampling_and_storage/?tab=java#sampling-rules
        samplingPriority: "AUTO_KEEP",
        // Default tags. They will be added into all span tags.
        defaultTags: null,
        // Custom Datadog Tracer options. More info: https://datadog.github.io/dd-trace-js/#tracer-settings
        tracerOptions: null
    }
}

call an API through moleculer-web

Context

Please provide any relevant information about your setup. This is important in case the issue is not reproducible except for under certain conditions.

Moleculer version: 0.14.12
Moleculer-web version: 0.9.1
NodeJS version: v14.16.0
Operating System: Arch Linux x86_64

Failure Logs

AndreMaz commented 3 years ago

@imatefx what's the version of dd-trace-js that you're using? Can you try the same version as in Moleculer's unit tests ("dd-trace": "^0.29.1") and see if it works?

imatefx commented 3 years ago

I was using "dd-trace": "^0.31.1" , tried it using "dd-trace": "^0.29.1" , but I am still getting the error


 TypeError: Cannot read property '94760' of undefined
    at DatadogTraceExporter.spanStarted (/home/stalin/dev/backend-service/node_modules/moleculer/src/tracing/exporters/datadog.js:158:38)
    at /home/stalin/dev/backend-service/node_modules/moleculer/src/tracing/tracer.js:219:55
    at Array.forEach (<anonymous>)
    at Tracer.invokeExporter (/home/stalin/dev/backend-service/node_modules/moleculer/src/tracing/tracer.js:219:18)
    at Tracer.spanStarted (/home/stalin/dev/backend-service/node_modules/moleculer/src/tracing/tracer.js:300:9)
    at Span.start (/home/stalin/dev/backend-service/node_modules/moleculer/src/tracing/span.js:87:15)
    at Tracer.startSpan (/home/stalin/dev/backend-service/node_modules/moleculer/src/tracing/tracer.js:205:8)
    at Context.startSpan (/home/stalin/dev/backend-service/node_modules/moleculer/src/context.js:412:30)
    at ServiceBroker.tracingLocalActionMiddleware (/home/stalin/dev/backend-service/node_modules/moleculer/src/middlewares/tracing.js:84:22)
    at metricsMiddleware (/home/stalin/dev/backend-service/node_modules/moleculer/src/middlewares/metrics.js:27:11)
    at Object.actions.<computed> [as rest] (/home/stalin/dev/backend-service/node_modules/moleculer/src/service.js:157:13)
    at Service.httpHandler (/home/stalin/dev/backend-service/node_modules/moleculer-web/src/index.js:343:39)
    at Server.emit (events.js:315:20)
    at /home/stalin/dev/backend-service/node_modules/dd-trace/packages/datadog-plugin-http/src/server.js:13:23
    at /home/stalin/dev/backend-service/node_modules/dd-trace/packages/dd-trace/src/plugins/util/web.js:74:60
    at async_hooks.js:313:14

But the _destroy function still exists if I initialize the spans object I get this error

TypeError: this.ddScope._destroy is not a function
    at DatadogTraceExporter.spanFinished (/home/stalin/dev/backend-service/node_modules/moleculer/src/tracing/exporters/datadog.js:196:17)
    at /home/stalin/dev/backend-service/node_modules/moleculer/src/tracing/tracer.js:219:55
    at Array.forEach (<anonymous>)
    at Tracer.invokeExporter (/home/stalin/dev/backend-service/node_modules/moleculer/src/tracing/tracer.js:219:18)
    at Tracer.spanFinished (/home/stalin/dev/backend-service/node_modules/moleculer/src/tracing/tracer.js:313:9)
    at Span.finish (/home/stalin/dev/backend-service/node_modules/moleculer/src/tracing/span.js:157:15)
    at Context.finishSpan (/home/stalin/dev/backend-service/node_modules/moleculer/src/context.js:430:8)
    at /home/stalin/dev/backend-service/node_modules/moleculer/src/middlewares/tracing.js:115:10
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async Worker.processJob (/home/stalin/dev/backend-service/node_modules/bullmq/dist/classes/worker.js:233:28)
    at async Worker.run (/home/stalin/dev/backend-service/node_modules/bullmq/dist/classes/worker.js:99:34)

But the _destroy function is there in dd-trace library

async_hooks.js#L134

The tests are passing even with the latest dd-trace library

OutdatedVersion commented 3 years ago

What if you force dd-trace-js to use the async_hooks API, @imatefx? Our team ran into similar as v0.27.0 switched the default API used to determine scope. To note, this would be a temporary patch until Moleculer is adjusted to fully support the changes.

{
    type: "Datadog",
    options: {
        // Datadog Agent URL
        agentUrl:
            process.env.DD_AGENT_URL,
        // Environment variable
        env: process.env.DD_ENVIRONMENT || null,
        // Sampling priority. More info: https://docs.datadoghq.com/tracing/guide/trace_sampling_and_storage/?tab=java#sampling-rules
        samplingPriority: "AUTO_KEEP",
        // Default tags. They will be added into all span tags.
        defaultTags: null,
        // Custom Datadog Tracer options. More info: https://datadog.github.io/dd-trace-js/#tracer-settings
-       tracerOptions: null
+       tracerOptions: {
+        scope: 'async_hooks',
+       }
    }
}

moleculerjs / moleculer