Configurable Log Output

[ABodycombe]

We are using NotGovUK on a project, and have a requirement for log output in a configurable JSON format, compatible with our ELK stack. For example:

{
  "timestamp": "2022-03-17T08:45:50,173Z",
  "appName": "my-nice-app",
  "level": "INFO",
  "x-correlation-id": "275d20d6-7bdd-426f-b8b3-bb69665b8813",
  "event_id": "SOME_EVENT",
  "message": "Some detail about this event"
}

The NotGovUK Engine doesn't provide any options for configuring the log output and the current output from bunyan doesn't meet our needs. The log level from bunyan is reported as a number, whereas the required format is a string.

There's also a requirement to output log information to the file system, with the log files sent to an external logging system.

For our current project, it may be sufficient to disable in in-built logger via a configuration parameter, and implement the required logging within the server index.ts. Alternatively, the NotGovUK engine could allow configuration of the bunyan streams and serializers to control the output.

[daniel-ac-martin]

Thanks for raising this @ABodycombe.

It would help me if you could look into whether Bunyan can be made to do what you want. (Otherwise the solution might have to be to provide your own logger in src/server/index.ts.)

With regards to the text description of the log level. Can that be in a different parameter or does it need to be level? e.g.

{
  "level": 30
  "levelText": "INFO"
  "message": "foo"
}

Could logging to a file could be achieved by piping the output?

I think it would definitely be good to allow setting the log-level so that it can be made more quiet. Do you not want any logging of requests?

[daniel-ac-martin]

If you can pipe the output, I wonder if the place to solve this problem might be there? i.e. Manipulate Bunyan's JSON into what you want.

[ABodycombe]

We definitely require a property named "level" which contains text. Inbound request logging isn't necessarily required, there's an HTTP Proxy which terminates the SSL and keeps an access log.

Disabling the bunyan logger is the simplest solution for us. We can log the outbound HTTP requests and responses in the correct format, without having to worry about re-formatting the bunyan log messages.

[daniel-ac-martin]

@ABodycombe: The next version will allow you to:

1. set the log level (so you could set this high to eliminate standard logging)
2. log to a file

But I don't yet have a way to set the level attribute to text, rather than a number.

Will that be enough for you?

[daniel-ac-martin]

@ABodycombe: This has now been released. Can you try upgrading to v0.4.2 after reading the release notes?

[daniel-ac-martin]

@ABodycombe: This functionality is now also available in the Home Office Design System in v0.2.1: https://github.com/UKHomeOffice/design-system/releases/tag/v0.2.0

(Sorry for the delay.)

[ABodycombe]

@daniel-ac-martin We've tested version 0.4.2, these logging changes do exactly what we need. Thanks for the speedy response on this.