[FEA] Support pytest warnings capture

[mdemoret-nv]

Currently, our C++/Python logger is not captured by Pytest and shown as part of the output. This can hide necessary warnings or cause developers to use two different systems for showing warnings. An example output is shown below:

With cuml.common.logger.warn()

======================================================================================= warnings summary ========================================================================================
<frozen importlib._bootstrap>:219
<frozen importlib._bootstrap>:219
  <frozen importlib._bootstrap>:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject
-- Docs: https://docs.pytest.org/en/stable/warnings.html
=============================================================================== 108 skipped, 2 warnings in 0.37s ================================================================================

With warnings.warn()

======================================================================================= warnings summary ========================================================================================
<frozen importlib._bootstrap>:219
<frozen importlib._bootstrap>:219
  <frozen importlib._bootstrap>:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject
cuml/common/import_utils.py:75
  /rapids/cuml-dev2/python/cuml/common/import_utils.py:75: UserWarning: The XGBoost library was found but raised an exception during import. Import Error:
  XGBoost Library (libxgboost.so) could not be loaded.
  Likely causes:
    * OpenMP runtime is not installed (vcomp140.dll or libgomp-1.dll for Windows, libomp.dylib for Mac OSX, libgomp.so for Linux and other UNIX-like OSes). Mac OSX users: Run `brew install libomp` to install OpenMP runtime.
    * You are running 32-bit Python on a 64-bit OS
  Error message(s): ["/usr/lib64/libm.so.6: version `GLIBC_2.23' not found (required by /opt/conda/envs/rapids/lib/libxgboost.so)"]
    warnings.warn("The XGBoost library was found but raised an exception during import. Import Error:\n{}".format(str(ex)))
-- Docs: https://docs.pytest.org/en/stable/warnings.html
=============================================================================== 108 skipped, 3 warnings in 0.37s ================================================================================

Part of this issue should be determining what about the warnings causes pytest to capture them, and adding that functionality to cuml.common.logger. PyTest references the following link: https://docs.pytest.org/en/stable/warnings.html. Tests should be added to ensure that warning filtering works as well.

[wphicks]

For awhile, I've been quite tempted to update this callback so that it uses a Python logger for output rather than just a print statement. That logger would do no formatting whatsoever, leaving spdlog responsible for formatting but also direct messages to the correct Python logging level, which would allow us to play nice with anything that handles or redirects Python logging (e.g. pytest).

The only downside to this is that we would then have three different places at which we could control the logging level: In the spdlog logger, in the Python logger, and in the handler for the Python logger. If we set the spdlog level to error and then the Python logger's level to debug, we would still not see debug messages, which might be counterintuitive. Nevertheless, it would solve the problem raised here.

[github-actions[bot]]

This issue has been marked rotten due to no recent activity in the past 90d. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

[github-actions[bot]]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.