Open kongzii opened 2 months ago
For some reason patching isn't working correctly in our API pods and some warnings
messages are still shown as errors in GCP.
To stop the fake alerts, I added this condition to the alerting policy:
NOT "sys:1: ResourceWarning: unclosed <ssl.SSLSocket"
The patch_logger function is starting to be quite complicated.
Could it be somehow simpler?
Currently, the main complication is that (1) GCP needs a specific format for logs to be parsed correctly and (2) if something is logging multi-line messages, it breaks the formatting so we have to replace all new lines with a space.
Another complication is that there are 4 different logging modules, we like loguru, libraries usually uses just Python's logging module, warning library is sometimes used instead of logging.warning method and a lot of 3-rd party libraries just go with a simple
print
, so we need to patch all of these.Theoretically, we could (hopefully somehow) re-configure the default behaviour in GCP Logs, where unformatted messages would be INFO instead of ERROR by default. But is that a good approach? (theoretically we could miss some important errors)