Error/warning summary at the end of pdf generation

[cweiske]

I'm converting a 54 page book into PDF and get lots of output. Many of it is debug output, but here and there are warnings between the hundreds of output lines.

It would be nice if there was a summary line that told how many errors and warnings there were, and in which categories the warnings fell.

Example:

Summary: 4 errors, 23 warnings
- 4x font error
- 12x font warning
- 68x passepartout
- 2x clipart
- 1x clipart color substitution 

[AnEnglishmanInNorway]

Nice idea. Getting the logging level right is tricky in any project. Another related thought might be a finer granularity in the logging. Clipart is a topic we could introduce. Then when the number of clipart related messages in the summary was high that would suggest that the clipart logging level could be increased. I didn't realise that passepartout was used much. Useful to know.

On Mon, 27 May 2024, 20:51 Christian Weiske, @.***> wrote:

I'm converting a 54 page book into PDF and get lots of output. Many of it is debug output, but here and there are warnings between the hundreds of output lines.

It would be nice if there was a summary line that told how many errors and warnings there were, and in which categories the warnings fell.

Example:

Summary: 4 errors, 23 warnings

• 4x font error
• 12x font warning
• 68x passepartout
• 2x clipart
• 1x clipart color substitution

— Reply to this email directly, view it on GitHub https://github.com/bash0/cewe2pdf/issues/147, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADBIDKM7QVCWIYYNDD5O4STZEN6CTAVCNFSM6AAAAABILT3PUGVHI2DSMVQWIX3LMV43ASLTON2WKOZSGMYTSNRUGU4TSNA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[AnEnglishmanInNorway]

I wonder if the best strategy here might be to build a "counting" wrapper around logging and use that everywhere we use logging.whatever. I went looking for that and found what appears to be a solution here, using handlers. With not much time before I am away from my machine again for a couple of weeks, I'll leave what I discovered here, in case anybody wants to follow up. First, the counting error handler

# based on Rolf's answer in https://stackoverflow.com/questions/812477/how-many-times-was-logging-error-called
import logging
class MsgCounterHandler(logging.Handler):
    levelToCountDict = None

    def __init__(self, *args, **kwargs):
        super(MsgCounterHandler, self).__init__(*args, **kwargs)
        self.levelToCountDict = {}

    def emit(self, record):
        l = record.levelname
        if (l not in self.levelToCountDict):
            self.levelToCountDict[l] = 0
        self.levelToCountDict[l] += 1

Then I added some test code just after the creation of the configlogger in cewe2pdf.py

configlogger = logging.getLogger("cewe2pdf.config")
from messageCounterHandler import MsgCounterHandler
countingHandler = MsgCounterHandler()
countingFormat = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
countingHandler.setFormatter(countingFormat)
countingHandler.setLevel(logging.DEBUG)
configlogger.addHandler(countingHandler)
configlogger.warning("counting an warning message")
configlogger.info("counting an info message")
configlogger.warning("counting an warning message")
configlogger.debug("counting a debug message")
configlogger.error("counting an error message")
print(countingHandler.levelToCountDict)

Running that produces {'WARNING': 2, 'ERROR': 1}

which is not quite what we want. It doesn't count the info and debug level messages, presumably because the configlogger is set to output at warning level. That wasn't what I was expecting (I copied from an example which said that each handler has its own level setting) but I suppose it makes some sense. To get what we want we must to specify the configlogger to output at DEBUG level, so it lets everything through, and then we configure each handler that we're going to use for the configlogger individually. The code above configures the counting handler to register everything all the way down to DEBUG messages. Then we modify the yaml file so that the configlogger also lets everything through, but we give it its own console logger which is configured to only let through WARNING or higher. handlers:

  console:
    class: logging.StreamHandler
    level: DEBUG
    formatter: simple
    stream: ext://sys.stdout
  consoleForConfig:
    class: logging.StreamHandler
    level: WARNING
    formatter: simple
    stream: ext://sys.stdout
loggers:
  cewe2pdf.config:
    handlers: [consoleForConfig]
    level: DEBUG
    propagate: no

Running that gives us what we want from my little test: {'WARNING': 2, 'INFO': 1, 'DEBUG': 1, 'ERROR': 1}

So that's the principle established. If nobody else picks this up, I'll take it in a few weeks time.