cal-adapt / climakitae

A Python toolkit for retrieving, visualizing, and performing scientific analyses with data from the Cal-Adapt Analytics Engine.
https://climakitae.readthedocs.io
BSD 3-Clause "New" or "Revised" License
21 stars 2 forks source link

Creating Application Logger + ProgressBar for Data Loading #337

Closed claalmve closed 7 months ago

claalmve commented 7 months ago

Description of PR

Creating initial applocation to increase visibility in function calls in cae-notebooks. Additionally, added a Dask ProgressBar around computing data to memory so the users is able to see the progress of their loading/computing dataset.

Initial loggers are added in agnostic.py to demonstrate its usage, but we can discuss other places to put loggers throughout the codebase as seen fit, in a future iteration.

Summary of changes and related issue We want users to have more visibility of what's happening behind the scenes.

Type of change

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce.

# If you're interested in what's going on in the back-end, you can `enable_app_logging` and `disable_app_logging` as you'd like!
from climakitae.util.logging import enable_app_logging, disable_app_logging
enable_app_logging()
disable_app_logging()

Checklist:

Tianchi-Liu commented 7 months ago

I tested this in agnostic.ipynb. The progress bar is looking good, but I am not seeing the logger messages: image

Could you add instructions on how the users can turn the messages on?

Tianchi-Liu commented 7 months ago

The progress bar for ck.load() is awesome! I noticed that ck.load() no longer outputs this message: “Processing data to read xxx MB of data into memory... “ I think with the progress bar implemented, the message no longer essential, but want to note a small caveat that the progress bar might not make sense if it appears by itself in the middle of a long log, e.g. for batch mode. It would be great if there’s an easy fix, but such edge cases could also be addressed separately with print statements at the very least.