Add custom token data to @elastic/ecs-morgan-format

[sanhardik]

Hello, I would like to add custom data so that it gets added to the final formatter for @elastic/ecs-morgan-format

Is it possible ?

[trentm]

@sanhardik It isn't possible with the current implementation. Is the data you want to add static? I.e. it doesn't change for the lifetime of the app process? If so we could possibly add a feature so that something like this would work:

const app = require('express')()
const morgan = require('morgan')
const ecsFormat = require('@elastic/ecs-morgan-format')

app.use(morgan(ecsFormat({
  fields: {foo: 'bar'}
})))

to add foo: 'bar' to every log record.

[sanhardik]

@trentm Thanks for the reply. The data wont be static. The information that I am thinking, is adding more details about the request. In my case, I am dealing with API calls about devices (Edit,Operate etc)

I want to add information about the device whose resources are being accessed.

I was thinking of extending the request or the response object and then passing that on. May if I could pass a "Morgan Token" in, to be used in final JSON string

const app = require('express')()
const morgan = require('morgan')
const ecsFormat = require('@elastic/ecs-morgan-format')

morgan.token('device', function (req, res) { return req.device })

app.use(morgan(ecsFormat({
  tokens: ['device']
})))

[trentm]

I think something like that would be reasonable to implement. I haven't played with using morgan tokens, so I don't know for sure.

I most likely won't have time soon to play with this. Would you be willing to try making a patch for this?

[jpage-godaddy]

I have a pull request proposal for allowing calculated custom fields to be injected.

https://github.com/elastic/ecs-logging-nodejs/pull/179

[SokolovAlexanderV]

Hello. I've found a way to add custom field to ECS formatted log data without @elastic/ecs-morgan-format. The trick is to use Morgan "custom format function" which does not return format string but logs request and response with Winston logger which uses ECS format. It's a bit more complex than sanhardik's code but it solves the problem.

const express = require('express');
const morgan = require('morgan');
const winston = require('winston');
const ecsFormat = require('@elastic/ecs-winston-format');

const app = express();

/*
    Create Winston logger
    either attach it to app to use within request callbacks or
    define a constant: const logger = ...
*/
app.logger = winston.createLogger({
    format: ecsFormat({
        convertReqRes: true // this option is IMPORTANT
    }),
    transports: [ new winston.transports.Console({
        level: 'debug' // this is IMPORTANT if you want debug level
    }) ],
});

/* Configure Morgan middleware but DO NOT RETURN value */
const accessLogMiddleware = morgan((tokens, req, res) => {

    /* Works finne with convertReqRes option set to true */
    let params = { req, res };

    /* Here is "device" field */
    params.device = req.device || 'No device';

    /* Log formatted message with extra params instead of returning format string */
    const level = res.statusCode < 500 ? 'info' : 'error';
    const msg = morgan.compile(morgan['combined'])(tokens, req, res);
    app.logger.log(level, msg, params);
});

app.use(accessLogMiddleware);

/* Some body data can be also used to log with request  */
app.use(express.json());
app.use(express.urlencoded({ extended: true }));

app.get('/', (req, res) => {
    req.app.logger.log('debug', 'GET request');
    res.send('Get request, no body');
});

app.post('/', (req, res) => {
    req.app.logger.log('debug', 'POST request', {extra: 'data'});
    res.send('Post request, has body');
});

app.listen(8080, () => {
    app.logger.info('Express application is listening on port http://127.0.0.1:8080 ...');
});