search

Financial-Times / splunk-heroku

Support for Heroku log drains packaged as a Splunk app.
https://tech.in.ft.com/tech-topics/logging/splunk/logging-from-heroku
MIT License
2 stars 0 forks source link

Anonymize data #37

Open sjparkinson opened 2 years ago

sjparkinson commented 2 years ago

Here's the relevant Splunk documentation, https://docs.splunk.com/Documentation/Splunk/9.0.0/Data/Anonymizedata.

We've recently ended up needing to delete PII (email addresses) from some logs in the heroku index where a system was logging all details about some POST data (correct me if I'm wrong here).

Should we consider proactively anonymizing data using the heroku source type? If so what should we be looking for?

I suspect looking back of previous incidents requiring data deletion would be insightful!

sjparkinson commented 2 years ago

@rowanmanning we discussed moving across some log sanitization. You mentioned it's already being done within apps, I had a quick look at n-logger but couldn't find anything. Let me know where to look and I can start to build up a set of rules in this issue.

rowanmanning commented 2 years ago

n-mask-logger is the main place we do anything, and it seems to mostly be checking for email-address-like values and restricting based on field name rather than value