search

kdgregory / log4j-aws-appenders

Appenders for Log4J 1.2.x, Log4J 2.x, and Logback that write to AWS destinations.
Apache License 2.0
67 stars 18 forks source link

Unable to use LogEventPatternConverter with the appender #201

Closed MuthiahPrabhakaran closed 4 months ago

MuthiahPrabhakaran commented 4 months ago

Hi,

I am working on a spring boot project. The project uses log4j.xml to log the events into a file. And the log4j.xml uses your kinesis appender inside it and configured to redirect the INFO level logs to the Kinesis stream. No issues here.

Recently, I had to mask PCI/PII data. To do that, I used LogEventPatternConverter and overridden the logs with the specific regex pattern. Sample code can be found here. This is also working good.

But, when the log gets redirected to Kinesis, the masking is not happening. How to add the converter key into the message field of the KeyValuePair? I have raised a stackoverflow question also. You can have a look for more details.

kdgregory commented 4 months ago

@MuthiahPrabhakaran

You'll need to create a complete, minimal example that shows the problem.

The blog post that you've linked shows a converter used with PatternLayout, while your Stack Overflow post shows the example code from this library, which uses a JsonLayout. I would not expect that a plugin designed for one would work for the other.

MuthiahPrabhakaran commented 4 months ago

@kdgregory I will share the actual file in a while. To mask the sensitive data on a project level(not on the file level), I used PatternLayout and implemented LogEventPatternConverter. It works when the log is being written to a file on the server. On the other hand, I used Kinesis appender with JsonLayout to redirect the logs to Kinesis with some custom keyValuePair. When the log is getting redirected to the Kinesis, the data is not masked. Simply put, I use PatternLayout and JsonLayout for a different purpose. So the question is, is there any way to enforce the Kinesis appender to redirect the log after the log went through the LogEventPatternConverter?

kdgregory commented 4 months ago

No. Appenders -- any appender, not just the ones from this library -- call the layout to format the message, then send it on to the destination without any further changes.

If you want to redact fields in a JsonLayout, you'll either need to find a plugin that does that, or extend JsonLayout yourself.