Use pino logMethod hook to format audit logs

[nicola88]

Feature proposal

Use pino logMethod hook to enrich audit logs with basic fields, like version, timestamp and checksum.

The hook would filter the audit logs based on the level (greater than 1000) and wrap them into a standard field to ensure audit logs coming from different services follow the same data model.

Feature description

All services generating audit logs need to follow a common data model to ensure end users can aggregate and query them in a unified way.

We propose to use pino logMethod hook to intercept audit logs based on their log level, wrap the object passed as first argument to the log method and enrich it with some computed fields (version, timestamp, checksum, etc.).

Feature snippet example

The following snippet provides an example of how we imagined to configure pino to generate audit logs.

const { createHash } = require('node:crypto')
const pino = require('pino')

const options = {
  customLevels: {
    audit: 1100,
  },
  hooks: {
    logMethod(inputArgs, method, level) {
      if (level > 1000 && inputArgs.length >= 2) {
        const object = inputArgs.shift()
        const auditObject = {
          auditEvent: {
            version: '1.0.0',
            timestamp: new Date().toISOString(),
            checksum: {
              algorithm: 'sha512',
              value: createHash('sha512')
                .update(JSON.stringify(object))
                .digest('hex'),
            },
            metadata: object,
          },
        }
        return method.apply(this, [auditObject, ...inputArgs])
      }
      return method.apply(this, inputArgs)
    },
  },
}

const logger = pino(options)

const metadata = {
  event: 'AM/AppointmentCreated/v1',
  resource: 'AM/Appointment/appointment-12345',
  user: 'auth0|dr.john.doe',
  operation: 'CRUD/POST',
  source: 'appointment-manager',
}

logger.audit(metadata, 'event')

[fredmaggiowski]

Hi, as for #344 I'd not bind the library to the concept of custom logging methods (e.g. audit), however providing customized log hooks from your module in order to make you capable of doing what you are proposing makes completely sense to me, can you submit a pr?

[nicola88]

Thanks, we will open a PR soon to expose the custom hooks.

We were also wondering how we could avoid duplicating the hook implementation code across all projects generating audit logs, since it would become cumbersome to maintain over time and also quite bad in terms of developer experience, as Giulio pointed out (thinking about customers that want to use audit logs in their services).

Do you have any suggestion how we could avoid this? Maybe having in lc39 or the custom plugin lib a collection of ready-to-use hooks? Or a dedicated configuration option to enable audit trail, that will both add the custom log level and the hook? That would also make easier to filter the log not simply assuming their level to be greater than a thousand (level > 1000 && inputArgs.length >= 2), but checking that they match exactly the chosen log level (level === 1100 && ... from the snippet).

[fredmaggiowski]

We were also wondering how we could avoid duplicating the hook implementation code across all projects generating audit logs, since it would become cumbersome to maintain over time and also quite bad in terms of developer experience, as Giulio pointed out (thinking about customers that want to use audit logs in their services).

I'd create a specific template for services that may need audit or a supporting lib that provides the log definitions to be supplied to lc39.

I'd not add custom logics to lc39 (and neither to the cplib)

[nicola88]

Make sense, we can integrate our custom hook and also the related documentation in the Mia Care Node.js template. I close this issue and will open the PR in the next days.

[nicola88]

I opened the PR https://github.com/mia-platform/lc39/pull/366