search

ankane / logstop

Keep personal data out of your logs
MIT License
240 stars 13 forks source link

Url encoded emails not filtered #2

Closed jonian closed 5 years ago

jonian commented 5 years ago

Hi @ankane, url encoded emails are no being filtered. Example:

email%40example.com

I you want I can make a PR and change the regex:

# Current regex
EMAIL_REGEX = /\b[\w+\-.]+@[a-z\d\-]+(\.[a-z\d\-]+)*\.[a-z]+\b/i

# Regex with url encoded emails support
EMAIL_REGEX = /\b[\w+\-.]+(@|%40)[a-z\d\-]+(\.[a-z\d\-]+)*\.[a-z]+\b/i
ankane commented 5 years ago

Hey @jonian, I could see it possibly making sense to add this. What's the situation where you're seeing it?

jonian commented 5 years ago

Thank you for the quick response. I am seeing this when using typhoeus.

Log from the app that sends the request:

ETHON: performed EASY effective_url=http://localhost:5000/api/users/find?email=example%40test.com response_code=404 return_code=ok total_time=0.353005

Log form the app that receives the request:

Started GET "/api/users/find?email=example%40test.com" for ::1 at 2018-12-11 11:41:44 +0200
ankane commented 5 years ago

Cool, seems like enough of a reason to add it. Feel free to make a PR.

ankane commented 5 years ago

Added a section to the bottom of the readme with instructions on how to get set up.

ankane commented 5 years ago

Fixed by #3

jonian commented 5 years ago

Adding more info that I forgot to mention in previous comments.

This should be used in addition to config.filtered_parameters, not as a replacement.

Adding :email to config.filtered_parameters works with the app that receives the request but not the app that sends the request.

That is an issue with typhoeus/ethon#153.

ankane commented 5 years ago

Yeah, Logstop is there to catch you when you forget to add it to filtered_parameters.

Re Ethon: I don't think it has any concept of Rails since it's a Ruby library, so I wouldn't expect it to filter. Not sure if it has its own way to filter sensitive data, besides turning off logging.