Improve logging configuration

NASA-PDS / registry-sweepers

Scripts that run regularly on the registry database, to clean and consolidate information

Apache License 2.0

0 stars 1 forks source link

Improve logging configuration #47

Open tloubrieu-jpl opened 11 months ago

tloubrieu-jpl commented 11 months ago

💡 Description

[ ] use tqdm to log progress (if needed in an AWS context) see comment https://github.com/NASA-PDS/registry-sweepers/pull/45/files#r1275514146
[ ] externalize log configuration in a file (see comment https://github.com/NASA-PDS/registry-sweepers/pull/45/files#r1275520453)

alexdunnjpl commented 5 months ago

@tloubrieu-jpl why is tqdm actually desirable here? The intent is to show (in ECS logs) periodic progress of various long-runnning queries, which are being consumed concurrently. My gut feeling is that tqdm either wouldn't show up in the way we expect, or would spam the logs in a way that is less legible than the current approach, because tqdm relies on having an interactive terminal with backspacing capability, afaik.

tloubrieu-jpl commented 5 months ago

Hi @alexdunnjpl , none of the items in this ticket is really important.

The first one aims at replacing some ad hoc code with the off-the-self module tqdm of something else. The intention is to simplify the code. Maybe tqdm is not what we need... as you said I used it with interactive terminal or in julyter notebooks, I don't know if we can get something useful out of it for text logs.