search

oats-center / isoblue

ISOBlue Hardware, Avena software, and Deployment Files
MIT License
20 stars 9 forks source link

Python ENV based logger #56

Open aaron97neu opened 3 years ago

aaron97neu commented 3 years ago

Nodejs has a very nice ENV based logger called debug. This is especially helpful in our containers for writing logs to disk instead of having to chase them in stdout/err. We should find and use a similar library for our python based containers

aaron97neu commented 3 years ago

This might be what we are looking for: https://docs.python.org/3/library/logging.html

jarett-lee commented 3 years ago

Basic logging style using the logging library

gps2tsdb_1  | INFO:root:Initing Postgres obj

A nice feature would be to set the logging level from the Docker compose file either using strings or numbers.

Do we want to change the timestamp format in the logs? Do we want to compare the logging timestamp to the database timestamp?

abalmos commented 3 years ago

@jarett-lee I think we should set the logging level via env variables like this. That is the natural way to pass arguments into containers.

jarett-lee commented 3 years ago

Example output as of most recent changes:

gps2tsdb_1  | 2020-10-05 02:02:33.517 UTC INFO     Ensuring tables are setup properly
gps2tsdb_1  | 2020-10-05 02:02:33.518 UTC WARNING  Ensuring GPS point table is a timescaledb hypertable
gps2tsdb_1  | 2020-10-05 02:02:33.520 UTC ERROR    Finished settting up tables

I also added env support. It looks like basicConfig can handle strings on its own, but I had to put in some logic to handle integers.

aaron97neu commented 3 years ago

I think those changes satisfies what @abalmos was asking for? If so, I think we are close to being able to open the PR and start reviewing the code itself?

abalmos commented 3 years ago

Unfortunately I don't have the time to review at the moment, but anything along those lines is an improvement and seems worthy of opening a PR. It will also let us test how the actions deal with external PRs.

Out of curiosity, does this work like debug does? For example:

$ DEBUG="err:*,trace:sub-system-I-care-about:*" python my-broken-service.py

would output all error messages and only traces from "sub-system-I-care-about".

In short, debug just has a "tag" for each message and the typically pattern is to write the tags like "type:system:sub-system:sub-sub-system:..." and then the DEBUG evn is a pattern match on those tags.

BTW -- I'm not saying we should or should not make that work, I am just wondering what we have now.

aaron97neu commented 3 years ago

As far as I know that is not currently implemented. Is that a feature in the nodejs debug lib? It seems quite helpful but we are likely limited to the features included in the python logging lib. An external lib may support this but I am unaware of it and would add another dependency

abalmos commented 3 years ago

@aaron97neu Correct, that is essentially all that the node debug library does. I'm not familiar enough with the logging module or any other python logging library to know if that is a reasonable comparison or not.

aaron97neu commented 3 years ago

This looks to be the closest but def does not work out the box https://docs.python.org/3/library/logging.html#filter-objects.