exclude_regex isn't working as expected

[ne0c0de]

Hi

I have an log lines like this:

Oct 2 12:34:03 mfa-rest2 mfa.out.log 2020-10-02 09:34:02: GET / 200 0.984 ms - 21054 
Oct 2 12:34:28 mfa-rest2 mfa.out.log 2020-10-02 09:34:27: GET / 200 1.022 ms - 21054 
Oct 2 12:34:29 mfa-rest2 mfa.out.log 2020-10-02 09:34:27: GET / 200 1.008 ms - 21054 
Oct 2 12:34:29 mfa-rest2 mfa.out.log 2020-10-02 09:34:28: GET /health 200 0.937 ms - 41 
Oct 2 12:34:32 mfa-rest2 mfa.out.log 2020-10-02 09:34:30: GET /health 200 1.085 ms - 41 
Oct 2 12:34:33 mfa-rest2 mfa.out.log 2020-10-02 09:34:32: GET / 200 1.033 ms - 21054 
Oct 2 12:34:34 mfa-rest2 mfa.out.log 2020-10-02 09:34:32: GET / 200 1.020 ms - 21054 
Oct 2 12:34:34 mfa-rest2 mfa.out.log 2020-10-02 09:34:33: GET /health 200 0.983 ms - 41 
Oct 2 12:34:37 mfa-rest2 mfa.out.log 2020-10-02 09:34:35: GET /health 200 0.969 ms - 41 
Oct 2 12:34:38 mfa-rest2 mfa.out.log 2020-10-02 09:34:37: GET / 200 0.881 ms - 21054 
Oct 2 12:34:39 mfa-rest2 mfa.out.log 2020-10-02 09:34:37: GET / 200 0.882 ms - 21054 
Oct 2 12:34:39 mfa-rest2 mfa.out.log 2020-10-02 09:34:38: GET /health 200 0.954 ms - 41 
Oct 2 12:34:42 mfa-rest2 mfa.out.log 2020-10-02 09:34:40: GET /health 200 0.963 ms - 41 
Oct 2 12:34:43 mfa-rest2 mfa.out.log 2020-10-02 09:34:42: GET / 200 1.607 ms - 21054 
Oct 2 12:34:44 mfa-rest2 mfa.out.log 2020-10-02 09:34:42: GET / 200 0.886 ms - 21054 
Oct 2 12:34:44 mfa-rest2 mfa.out.log 2020-10-02 09:34:43: GET /health 200 6.375 ms - 41 
Oct 2 12:34:47 mfa-rest2 mfa.out.log 2020-10-02 09:34:45: GET /health 200 0.944 ms - 41 
Oct 2 12:34:48 mfa-rest2 mfa.out.log 2020-10-02 09:34:47: GET / 200 0.904 ms - 21054 
Oct 2 12:34:49 mfa-rest2 mfa.out.log 2020-10-02 09:34:47: GET / 200 0.877 ms - 21054 
Oct 2 12:34:49 mfa-rest2 mfa.out.log 2020-10-02 09:34:48: GET /health 200 0.956 ms - 41 
Oct 2 12:34:52 mfa-rest2 mfa.out.log 2020-10-02 09:34:50: GET /health 200 0.948 ms - 41 
Oct 2 12:34:53 mfa-rest2 mfa.out.log 2020-10-02 09:34:52: GET / 200 0.868 ms - 21054 
Oct 2 12:34:54 mfa-rest2 mfa.out.log 2020-10-02 09:34:52: GET / 200 1.018 ms - 21054 
Oct 2 12:34:54 mfa-rest2 mfa.out.log 2020-10-02 09:34:53: GET /health 200 1.089 ms - 41 
Oct 2 12:34:59 mfa-rest2 mfa.out.log 2020-10-02 09:34:58: GET /health 200 1.066 ms - 41 
Oct 2 12:35:02 mfa-rest2 mfa.out.log 2020-10-02 09:35:00: GET /health 200 1.084 ms - 41 
Oct 2 12:35:03 mfa-rest2 mfa.out.log 2020-10-02 09:35:02: GET / 200 1.054 ms - 21054 
Oct 2 12:35:04 mfa-rest2 mfa.out.log 2020-10-02 09:35:02: GET / 200 1.520 ms - 21054 
Oct 2 12:35:04 mfa-rest2 mfa.out.log 2020-10-02 09:35:03: GET /health 200 1.085 ms - 41 
Oct 2 12:35:07 mfa-rest2 mfa.out.log 2020-10-02 09:35:05: GET /health 200 1.082 ms - 41 
Oct 2 12:35:08 mfa-rest2 mfa.out.log 2020-10-02 09:35:07: GET / 200 1.027 ms - 21054 
Oct 2 12:35:08 mfa-rest2 mfa.out.log 2020-10-02 09:35:07: GET / 200 1.000 ms - 21054 
Oct 2 12:35:09 mfa-rest2 mfa.out.log 2020-10-02 09:35:08: GET /health 200 1.429 ms - 41 
Oct 2 12:35:12 mfa-rest2 mfa.out.log 2020-10-02 09:35:10: GET /health 200 1.081 ms - 41 
Oct 2 12:35:13 mfa-rest2 mfa.out.log 2020-10-02 09:35:12: GET / 200 1.014 ms - 21054 
Oct 2 12:35:13 mfa-rest2 mfa.out.log 2020-10-02 09:35:12: GET / 200 0.937 ms - 21054 
Oct 2 12:35:14 mfa-rest2 mfa.out.log 2020-10-02 09:35:13: GET /health 200 1.066 ms - 41 

I'm trying to use exclude lines including GET /health 200 and using this line in config file:

exclude_regex = .*GET \/health 200.*

But LogDNA still pushing these lines to log monitoring. When I tred to check the pattern in regex tool it seems that it's matching with that unwanted lines. I tried to use double quotes on beginning and end of the regex pattern but it didn't wotk either.

I also checked the code but couldn't find any lines that's caring this exclude filter.

Any idea what am I doing wrong?

[smusali]

Hi @ne0c0de,

Thanks for submitting an issue!

Can you please tell us which version of the LogDNA Agent on which supported OS you are using?

Thanks!

[ne0c0de]

Hello @smusali

Thanks for fast response.

I'm using the latest version of LogDNS where I installed it on CentOS 8 from the original repo.

I found a solution to my problem this this regex:

exclude_regex = .*(GET /health|GET / ).*

Whenever I use backslash for escaping, it doesn't match with the line. I don't know why :)

Also I want to make my regex pattern as case insensitive but i can't do it because i can't use / in begin and end so i couldn't add i at the end of pattern. how can I achieve this?

[smusali]

@ne0c0de, I think using (?i) will help you to achieve it according to this.