NCAS-CMS / cf-python

A CF-compliant Earth Science data analysis library
http://ncas-cms.github.io/cf-python
MIT License
120 stars 19 forks source link

Add logging (replace '_debug' & more) pre-parallelisation #37

Closed sadielbartholomew closed 4 years ago

sadielbartholomew commented 4 years ago

More compound/intensive processes such as partitioning & subspace processing have logic dotted about to print relevant info if _debug is set for debugging purposes. It would be better to replace these with dedicated logging (& then extend the logging over time to cover more logic & include more detail for highly-verbose cases).

I think we should add basic logging before we start optimisation work in earnest because configurable well-placed logging calls will allow us to see clearly the state & location of the code as it runs, & then we e.g. only need to apply a custom handler to keep track of each process set-up by mpi4py if we go with that as a/the solution.

Logging would be good for various other reasons, both towards development & user support:

My opinion is that Python's logging module would be the best tool to use, as it is simple & there can be seamless interfacing with logged info from key imported modules (e.g. ESMPy, NumPy).

sadielbartholomew commented 4 years ago

A few points to update on. We: