Feature Request? Record a log compatible with cutelog via Python script

[rahmant3]

I'm exploring using the Python logging module in an automated test system. I'd like to automatically start the logger in my Python tests and archive the log file. I'd like people to be able to download the log file after the test is completed and view it in a UI like cutelog. Is this possible?

[busimus]

If you're fine with saving your log files in JSON format by using a formatter like python-json-logger then it's certainly possible, though not currently - I wrote the code to support loading of streamed JSON objects that formatters like this create but never pushed it to the repo.

But if you want your logs to be "human readable" and still load them in cutelog, then you should probably write a translator from your log format to JSON.

[rahmant3]

The first option would be perfectly acceptable to me. I don't mind if the files are not human readable. If you have some code already written I could try from a branch, I don't mind being a bit of a guinea pig, I'm just experimenting right now anyway. Or I also don't mind trying to writing it on my own if you could help point me to what the logging format looks like?

[busimus]

Pushed the changes to the main branch. You can try them out like this:

pip install git+https://github.com/busimus/cutelog.git

Then install python-json-logger and run this:

import logging
from pythonjsonlogger import jsonlogger

logger = logging.getLogger()

logHandler = logging.FileHandler('log.json')
formatter = jsonlogger.JsonFormatter()
logHandler.setFormatter(formatter)
logger.addHandler(logHandler)

logger.warning('test', extra={'test': 123})
logger.warning('test second line')

Then try loading the created log.json file in cutelog. It should load fine, but I haven't looked into that formatter's configuration, it's likely some options are incompatible with my file loading code.

[rahmant3]

Not bad! So by default the file only has the "message" field.

{"message": "test", "test": 123}
{"message": "test second line"}
{"message": "test error"}

But you can declare more fields in the log like this:

formatter = jsonlogger.JsonFormatter('[%(levelname)8s] %(message)s %(module)s %(name)s %(filename)s %(asctime)')

Then you can get more info like the timestamp, log level, name.

{"levelname": "WARNING", "message": "test", "module": "test_log", "name": "root", "filename": "test_log.py", "asctime": "2023-06-20 09:56:38,094", "test": 123}
{"levelname": "WARNING", "message": "test second line", "module": "test_log", "name": "root", "filename": "test_log.py", "asctime": "2023-06-20 09:56:38,094"}
{"levelname": "ERROR", "message": "test error", "module": "test_log", "name": "root", "filename": "test_log.py", "asctime": "2023-06-20 09:56:38,094"}

The only problem is that the timestamp doesn't seem to be populated correctly (it just seems to have my current system time), but otherwise this looks great! Love the filtering capabilities.

---

Here is the code I used to generate the log:

import logging
from pythonjsonlogger import jsonlogger

logger = logging.getLogger()

logHandler = logging.FileHandler('log.json')
formatter = jsonlogger.JsonFormatter('[%(levelname)8s] %(message)s %(module)s %(name)s %(filename)s %(asctime)')
logHandler.setFormatter(formatter)
logger.addHandler(logHandler)

logger.warning('test', extra={'test': 123})
logger.warning('test second line')
logger.info('test info')
logger.debug('test debug')
logger.error('test error')

logger2 = logging.getLogger("lib.utils")
logger2.warning("warning test 2")
logger2.error("error test 2")

logger2 = logging.getLogger("lib.core")
logger2.warning("warning test 3")
logger2.error("error test 3")

[busimus]

To fix the time issue specify created field instead of asctime in the formatter.

[rahmant3]

Thank you! Using created works perfectly. I've tried it with bigger log files from my test system and it works great there too.

[rahmant3]

No rush on releasing, but it would be great if this were available by the end of the month.

[busimus]

New version is out.