Support structured logging for Vercel 'edge' functions

[tchief]

Hi @chasers, thanks for the product!

Setup

Next.js project with middleware deployed to Vercel with pino-logflare.

Problem

Attempt to use logger either directly or via new child inside middleware

logger.info({ name: 'abc' }, 'In middleware');
...
const currentLogger = logger.child({ source: 'middleware' });
currentLogger.info({ name: 'abc' }, 'In middleware');

lead to edge function crash with message

"message": "[EvalError: Code generation from strings disallowed for this context]",
"source": "edge",
"type": "fatal"

Using console.log does not parse message into parsedLambdaMessage or similar property inside middleware.

Questions

1. Can I setup logger in a way it supports structured logging inside edge function?
2. Do you plan to include property similar to 'parsedLambdaMessage' for Vercel's edge functions?

[chasers]

Vercel middleware is a Cloudflare worker. So you’ll actually want to take our worker and adapt it accordingly. There shouldn’t be too much that needs changing structurally.

https://gist.github.com/chasers/c7a220e91820a1084b27fcfdb18ad6bd

On Wed, Jan 26, 2022 at 4:36 AM Sergii Golyshev @.***> wrote:

Hi @chasers https://github.com/chasers, thanks for the product! Setup

Next.js project with middleware https://vercel.com/docs/concepts/functions/edge-functions#middleware deployed to Vercel with pino-logflare https://github.com/Logflare/pino-logflare/blob/master/docs/VERCEL.md. Problem

Attempt to use logger either directly or via new child inside middleware

logger.info({ name: 'abc' }, 'In middleware'); ... const currentLogger = logger.child({ source: 'middleware' }); currentLogger.info({ name: 'abc' }, 'In middleware');

lead to edge function crash with message

"message": "[EvalError: Code generation from strings disallowed for this context]",

Using console.log does not parse message into parsedLambdaMessage or similar property inside middleware. Questions

1. Can I setup logger in a way it supports structured logging inside edge function?
2. Do you plan to include property similar to 'parsedLambdaMessage' for Vercel's edge functions?

— Reply to this email directly, view it on GitHub https://github.com/Logflare/pino-logflare/issues/51, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHY7JU5GMXZLP7ESBXUVIDUX7MCVANCNFSM5M2ZPQRA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

-- Logger.info("Email sent", %{ from: %{ name: "Chase Granberry", domain: "logflare.app", email: @.***" } })

[chasers]

And yeah if you're looking for data specifically from the middleware this is the way to do it. I'm happy to help assist you in modifying the Cloudflare worker accordingly.

[kjetilhartveit]

If you're using the API then i reverse-engineered createWriteStream and wrote a somewhat dirty way to support it in edge runtimes. I also had to install the npm package logflare-transport-core to get LogflareHttpClient.

Hopefully in the future edge will be supported out-of-the-box but this seems to work for me for now and I haven't found a subsitute for pino.

Update:) it seems pino will log as in inside the browser when in the edge runtimes (because native node is not supported). This is a bit annoying but it seems to work somewhat. I see the logged keys are different from the node version, e.g. keys defined in the base property in the pino configuration are missing (a bug?). Not quite sure what to do about that, the node and browser implementations seems completely different. My implementation should prevent a compile error on the edge and support both runtimes.

import {
    addLogflareTransformDirectives,
    handlePreparePayload,
} from "pino-logflare/dist/utils";
import { type Options } from "pino-logflare/dist/httpStream";
import { LogflareHttpClient } from "logflare-transport-core";

const stream = (() => {
    const options = {
        apiKey: /* YOUR API KEY */,
        sourceToken:  /* YOUR SOURCE */,
    } satisfies Options;
    return {
        /**
         * createWriteStream in pino-logflare re-written to work in Edge and ESM.
         *
         * The logic is simplified and copied from pino-logflare's repository.
         * Note that the implementation below can become stale from the original repository.
         * 
         * Sources:
         * - https://github.com/Logflare/pino-logflare/blob/master/src/httpStream.ts
         * - https://github.com/Logflare/pino-logflare/blob/master/src/streams.ts
         */
        write(msg: string, _?: string, callback?: (...args: any[]) => any) {
            const obj = JSON.parse(msg);
            const entry = handlePreparePayload(obj, options);
            const maybeWithTransforms = addLogflareTransformDirectives(
                entry,
                options,
            );
            const httpClient = new LogflareHttpClient(options);
            httpClient
                .addLogEvent(maybeWithTransforms)
                .then(() => {
                    callback?.(null);
                })
                .catch(callback);
        },
    };
})();