ing-bank / popmon

Monitor the stability of a Pandas or Spark dataframe ⚙︎
https://popmon.readthedocs.io/
MIT License
493 stars 33 forks source link

Change verbosity level #152

Open sbrugman opened 2 years ago

sbrugman commented 2 years ago

popmon by default provides logging and progress bar (tqdm) output while computing the report. There currently is no user-friendly way to change the verbosity level altogether.

rishabsinghh commented 1 year ago

Hey, i would like to start working on this issue. Can you please guide me?

sbrugman commented 1 year ago

@rishabsinghh Welcome!

The way logging is performed in popmon is that each Module has a logger property. Modules will call self.logger.info, self.logger.warn etc.

See for example:

In addition, the tqdm progress bar is used, e.g.

This issue was intended to re-evaluate the usefulness of each of the logging statements and their level (is it debug information? Then it should be self.logger.debug).

Then, the user should be able to control the level of output to receive (either via the logging config, or via popmon's Settings)

Does this help you forward?

ps. also open to use alive-progress instead of tqdm

rishabsinghh commented 1 year ago

Hey @sbrugman . This helps, i will ask you incase of any queries, thanks!

rishabsinghh commented 1 year ago

Hey @sbrugman . Few Thoughts! Please correct me if i am going in the wrong direction

  1. Verbosity can be defined as an input variables with numbers indicating the level of logging user needs. We may define this at the time we generate report settings.

Some thing like this!

image

And for progress bar similarly we can have another input variable called "show_progress_bar" which would be boolean. True and False.

Does this make sense?