search

zalando / nakadi

A distributed event bus that implements a RESTful API abstraction on top of Kafka-like queues
https://nakadi.io
MIT License
952 stars 292 forks source link

Fix logging leak #1469

Closed antban closed 1 year ago

antban commented 1 year ago

One-line summary

There is incorrect logging library usage - loggers are created for random names

Description

The loggers are supposed to be created per class or per other static entity. Right now Nakadi is creating loggers for random names. Logging libraries to not have an ability to remove loggers by names that are not in use anymore, and effectively this may result in OOM after some time.

This PR replaces the way how the subscription id, stream id and flow Id are passed and used in logging purposes.

  1. Nakadi usage of NDC is replaces with usage of MDC
  2. Subscription id and stream id are migrated to MDC instead of logger name
  3. A special filter for logger records is added to print only subscription ids we are interested in.

Deployment Notes

Logging configuration should be reworked

antban commented 1 year ago

@adyach @tor-vs-floki Please, take another look

adyach commented 1 year ago

👍

antban commented 1 year ago

:+1:

tor-vs-floki commented 1 year ago

👍

a1exsh commented 1 year ago

Noice!

Looks a bit involved though. Have we looked at putClosable, for example?