search

pravega / zookeeper-operator

Kubernetes Operator for Zookeeper
Apache License 2.0
368 stars 203 forks source link

Re-evaluate log level for zookeeper operator #318

Open anishakj opened 3 years ago

anishakj commented 3 years ago

Description

Reevaluate Log Level for Operator

Importance

Nice_to_have

Location

main.go , zookeepercluster_controller.go

Suggestions for an improvement

Use Zerolog to set log level for operator code and set various log levels

How to verify it

Deploy operator and zookeeper cluster with various log levels set and check if corresponding logs are displayed or not.

nosvalds commented 2 years ago

@anishakj I noticed this when trying to cut down my logging bills for Azure AKS.

Example log message

{"level":"info","ts":1660655236.650773,"logger":"controller_zookeepercluster","msg":"Updating StatefulSet","Request.Namespace":"default","Request.Name":"iati-prod-solrcloud-zookeeper","StatefulSet.Namespace":"default","StatefulSet.Name":"iati-prod-solrcloud-zookeeper"}

For example, I see around 1370/hr of "Updating Statefulset"

❯ k logs --since=1h solr-operator-zookeeper-operator-89db79877-rg47x | grep "Updating StatefulSet" | wc -l
    1371

Or looking at "level": "info", more than 9k/hr

❯ k logs --since=1h solr-operator-zookeeper-operator-89db79877-rg47x | grep 'level":"info' | wc -l
    9195

Would definitely appreciate a way to turn off the info logs, or reduce their frequency.

Thanks!

nosvalds commented 2 years ago

Note I believe upgrading to v0.2.4 which includes the fix for https://github.com/pravega/zookeeper-operator/issues/389 has reduced this greatly.

❯ k logs --since=1h solr-operator-zookeeper-operator-58cf9b8bb7-tq9sw | grep 'level":"info' | wc -l
     960