Logging level

[kevinkovalchik]

Hello,

I would like to be able to suppress the INFO logs that happen during training, but I have not been able to figure out how to do so. Setting verbose=False in the call to fit simply removes a progress bar, but does not remove the 100+ lines of logging.

For example:

[INFO kernel.cc:736] Start Yggdrasil model training
[INFO kernel.cc:737] Collect training examples
[INFO kernel.cc:392] Number of batches: 289
[INFO kernel.cc:393] Number of examples: 9224
[INFO kernel.cc:759] Dataset:
Number of records: 9224
Number of columns: 23
Number of columns by type:
    NUMERICAL: 22 (95.6522%)
    CATEGORICAL: 1 (4.34783%)

Columns:
NUMERICAL: 22 (95.6522%)
    0: "data:0.0" NUMERICAL mean:0.337444 min:0 max:1 sd:0.18369
    1: "data:0.1" NUMERICAL mean:0.337157 min:0 max:1 sd:0.183633
    2: "data:0.10" NUMERICAL mean:0.399254 min:0 max:1 sd:0.342231
...
...
...
[INFO abstract_model.cc:993] Engine "GradientBoostedTreesGeneric" built
[INFO kernel.cc:848] Use fast generic engine

For the model in question, there are 145 lines of this. The users of my code are generally biologists, and most of them will not understand nor want to see 145 lines of this, let alone see it multiple times if there is any optimization happening.

Is there a way to change the logging level here?

Thanks!

Best wishes, Kevin

[achoum]

Hi Kevin,

Can you tell me more about your setup? Notably, do you use colab, notebook or is your program a python script?

Background

Training logs are printed in the standard error channel.

In a colab, model.fit() will only print the keras progress bar. The training logs (the 145 lines you are mentioning) are normally not printed.

However, the training logs will be printed in the colab instance process (e.g. in the console if you run your own colab instance).

You can print the training logs in the notebook, by surrounding the fit call with with sys_pipes(): (this is not what you want).

If you use a python script directly (i.e. you don't use colab), the logs are printed in the (only) console. There is currently no nice way to disable logging in the console, as passing --alsologtostderr=false only work if you use the C++ API.

A temporary solution is to hide the standard error during training (see this discussion).

Actions

1. Make sure fit is not surrounded by with sys_pipes():.

2. Disable the stderr. For example:

from contextlib import contextmanager,redirect_stderr,redirect_stdout
from os import devnull

@contextmanager
def suppress_stdout_stderr():
    """A context manager that redirects stdout and stderr to devnull"""
    with open(devnull, 'w') as fnull:
        with redirect_stderr(fnull) as err, redirect_stdout(fnull) as out:
            yield (err, out)

with suppress_stdout_stderr():
    model.fit(train_ds)

[kevinkovalchik]

Thanks for the response. I am running things from the console, not colab or a notebook.

I did try to redirect the stderr to devnul without results, but I was changing sys.stderr directly. Using redirect_stderr with the context manager seems like a better, solution so I will try that, Also, I am using PyCharm, and their console sometimes seems to have some unexpected behavior at times, so if the above doesn't work I will try using the system console.

I'll let you know what happens,

[kevinkovalchik]

I tried redirecting the stderr but the training logs are still being printed to the console, which seems strange. I can confirm that the stdout is being redirected, so it seems the stderr must be as well, but somehow the compiled code is bypassing this? This is happening in the PyCharm console and the system console.

Here's more information on my setup:

• Pop! OS 21.04 (Ubuntu 21.04 derivative)
• tensorflow 2.7.0
• tensorflow-decision-forests 0.2.1

[achoum]

Noted. In this case, I'll expose logging control in the API. If all goes well, it will be part of the next release coming this month.

Cheers.

[achoum]

TLDR: The feature was added in TF-DF 0.2.2.

The amount of displayed training logs is controlled by verbose model constructor or fit argument. This parameter impacts both the training logs exported to the python's standard output and the C standard output and error.

verbose=0 => No logs. verbose=1 => High level training logs (default) verbose=2 => All the training logs