httpHandler should use W3C Context Tracing header to set context requestID and parentID

[synsteve]

I've been trying to get my multiple moleculer-web applications to produce a consistent distributed trace but have found a problem with parent spans.

Using the x-correlation-id or x-request-id header does work for the main trace id, but I can't set the parent span id (ctx.parentID) properly.

The span for the api.rest call is created before I can set the ctx.parentID

I tried to use a global middleware:

const parseTraceContext = (req) => {
    let trace = {};
    if (!req || !req.headers) return trace;
    if (req.headers["traceparent"]) {
        let traceparent = req.headers["traceparent"].toLowerCase();
        if (traceparent.match(/^([0-9a-f]{2})-([0-9a-f]{32})-([0-9a-f]{16})-([0-9a-f]{2})$/)) {
            [trace.version, trace.id, trace.parentSpan, trace.flags] = traceparent.split("-");
        }
    } else {
        // Look for X-B3-Traceid, X-B3-Spanid
        if (req.headers["x-b3-traceid"]) {
            trace.id = req.headers["x-b3-traceid"].toLowerCase();
        }
        if (req.headers["x-b3-spanid"]) {
            trace.span = req.headers["x-b3-spanid"].toLowerCase();
        }
        if (req.headers["x-b3-parentspanid"]) {
            trace.parentSpan = req.headers["x-b3-parentspanid"].toLowerCase();
        }
    }
    return trace;
};

module.exports = {
    sessionTracking: () => (req, res, next) => {
        const trace = parseTraceContext(req);
        if (trace.id) req.$ctx.requestID = trace.id;
        if (trace.span) req.$ctx.id = trace.span;
        if (trace.parentSpan) req.$ctx.parentID = trace.parentSpan;
        next();
    },
};

That looks for the appropriate header and sets the req.$ctx properties, but I think because when the rest action is called, the internal tracing middleware is called before the api middleware above, it doesn't give the api span a parentID.

The only way around this would be to either code for this directly in the httpHandler or provide for some callback before the call to the rest action in which I could detect the headers and set the parentID in the options passed to this.actions.rest

[icebob]

I will make a built-in function which can read these properties and use it as tracing IDs. Could you post the "official" specification about these headers?

[AndreMaz]

I think that this is the specification: https://www.w3.org/TR/trace-context/#traceparent-header

[synsteve]

Yep, thanks AndreMaz & icebob

There's a regex for it in my code above it it helps with the 4 sections being:

• version (always 00 AFAI can tell)
• traceID (requestID), 32 char hex
• parentID, 16 char hex
• flags - only one so far is "sampled" (bitmask 0x1). Not sure if you can weave this into moleculer's sampling (I think maybe setting ctx.tracing to true/false is the right this to do)

[icebob]

@synsteve I've done something :) Could you try it? Because I can't test it in a real environment with coming tracing data. You can define a rootCallOptions in service settings which can be an Object or a Function. In you case you need the Function which gets the req and you can fill the options which will be passed to the root api.rest action call as calling options. Here is an example for your use-case: https://github.com/moleculerjs/moleculer-web/blob/94591713e1975a26ee13041418ed07c717002e7f/examples/dev/index.js#L25-L51

Please try it with npm i moleculerjs/moleculer-web and if it's good I can release a new beta version.

[synsteve]

Hi @icebob - works a treat! I used your example code and then when I breakpoint in the onBeforeCall callback, ctx.parentID, ctx.requestID, ctx.span.parentID and ctx.tracing are all set to the correct values.

Nice job!

[synsteve]

Hi @icebob - has this made it into the beta yet?

Thanks.

[icebob]

Sorry, I forgot it. I've just released in https://github.com/moleculerjs/moleculer-web/releases/tag/v0.10.0-beta3