search

change-metrics / monocle

Monocle helps teams and individual to better organize daily duties and to detect anomalies in the way changes are produced and reviewed.
https://demo.changemetrics.io/
GNU Affero General Public License v3.0
362 stars 56 forks source link

Configure Log Level #1044

Closed SaschaLucius closed 1 year ago

SaschaLucius commented 1 year ago

Hey, I'm using monocle with kubernetes in an AWS Cloud environment. Since my load balancer is checking the health of the application frequently, I'm facing a lot of info log messages of the following kind: - - [25/May/2023:09:24:44 +0000] "GET /health HTTP/1.1" 200 - "" "ELB-HealthChecker/2.0"

Is there any way to configure the log level of the server monocle is using?

Have looked into the code, and I'm not sure how to do it. Maybe https://github.com/change-metrics/monocle/blob/master/k8s/api-service.yaml could be the entrance point.

TristanCacqueray commented 1 year ago

Hi, logging is presently not configurable, and I'm not sure what should be the options. Though I think it's ok to silence /health and /metrics request, as proposed in #1045 .

SaschaLucius commented 1 year ago

@TristanCacqueray thank you for your fast response and a proposed solution. My initial idea was to change the log level to warning instead of info to ignore these status messages. But your solution would also help to solve my problem.

Therefore, thank you I will have a sort look at your PR, but my Haskell skills are a little bit rusty

TristanCacqueray commented 1 year ago

You are welcome, and thank you for the feedback. I meant, having configurable log level sounds like a good feature to me, but we need to define how this would work. For example, HTTP logs are managed by the http server library (warp), and they don't have a severity attached. The monocle's backend code do use warning/info/debug levels. So I guess we could have:

We also talked about pushing the logs in elasticsearch too, that way the console output could only have warnings.

SaschaLucius commented 1 year ago

My question would be, are we already discussing premature optimization?

Your proposed fix helps me with my issue, so I'm happy.

E.g. the proposed fix for Nginx is to ignore dependent on the agent https://ismailyenigul.medium.com/disable-logging-on-nginx-pod-for-elb-healthchecker-2-0-user-agent-e745ad55a7b4 .

Future wise, I agree a global level setting or a component wise would be nice, but until someone else than me is complaining about it, I would not invest into it ;)