Improve Logging

[Solosneros]

Hello,

we are already using the default Python package for logging in PyVBMC. I believe that there are several open issues that need to be discussed.

In VBMC (MATLAB) the printing of messages is configured using the display option (see vbmc.m). In PyVBMC this has been done in a similar way (see vbmc.py). Logging messages are then logged using the logging levels (WARN/INFO/DEBUG). I believe that this part is ported in a pythonic way and does not need major refactoring.

Some details require refactoring in my opinion:

Output to stderr/stdout:

Currently, we initialize the root logger ourselves, when the user hasn't done that yet. That means by default we log to stdout, but the user can overwrite that logging setting before initializing PyVBMC. See the documentation of the baseConfig. We should give the user some kind of option to configure that.

Printing the column headers and plotting

In Jupiter notebooks when plotting of the iterations is enabled, we want to alternate between logging the column headers, the logging data and the plot of the current iterations. See the following picture:

There are edge cases where this does not work (e.g. when logging to a file the columns are always printed, even though it is not necessary). I think this is related to the output to stderr/stdout point.

Testing of logging

The logging has not been tested yet. We need to do that, but this seems tricky.

How many loggers should be there?

Currently, there is only one logger, but maybe it makes sense to split the PyVBMC logger into multiple ones.

Maybe there is more to discuss - but those are the major points.

[lacerbi]

Thanks for the comments and the analysis!

• Regarding output to stdout/stderr: my understanding is that we initialize the root logger only if the user has not already done it. So I presume we don't need to add any specific options? If the user wants the root logger to print to some different stream, they can initialize it themselves, and PyVBMC will follow, or is there any problem with this approach?
• Printing the column headers and plotting: I think we should simply add an option to PyVBMC about how the header printing works. By default, PyVBMC will try to guess the best choice, which is: if plot is on, print the header at every iteration; if plot is off, print only once on top. The user can overwrite this option if needed. The option could literally be print_iteration_header or something similar (default None which triggers the automated choice).
• Testing logging is important, we should think how to do it.
• Multiple loggers seem overkill for now.

[Solosneros]

Thanks for the reply, @lacerbi :)

• Regarding output to stdout/stderr: yes, sounds sensible.
• Printing the column headers and plotting: agree
• Testing: This especially becomes interesting and important when implementing the column header stuff.
• Multiple loggers: ok