Support for additional options without having to use syntax specific to StructuredLogger

[moqmar]

• PySeq version: 0.3.20
• Python version: 3.10.6
• Operating System: Windows

Description

I'm trying to add named log properties without having to use logger calls that only work with seqlog. This means that I can't use the following, which obviously works with seqlog, but fails without it:

logging.getLogger("Example").warning("Hello", Name="World")

As an alternative, I have tried the following:

logging.getLogger("Example").warning("Hello", extra={"Name": "World"})
logging.getLogger("Example").warning("Hello", extra={"log_props": {"Name": "World"}})

I'd have expected at least the latter to work, and would prefer the former to be includable as an "Extra" field in Seq, as I guess most extra information provided by libraries can be relevant.

What I Did

Python 3.10.6 | packaged by conda-forge | (main, Oct  7 2022, 20:14:50) [MSC v.1916 64 bit (AMD64)] on win32
>>> import logging
>>> logging.getLogger("Example").warning("Hello", extra={"Name": "World"})
Hello
>>> logging.getLogger("Example").warning("Hello", extra={"log_props": {"Name": "World"}})
Hello
>>> logging.getLogger("Example").warning("Hello", Name="World")
Traceback (most recent call last):
  File "C:\Users\XOMMARQU\Miniconda3\envs\iris_app\lib\code.py", line 90, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
  File "C:\Users\XOMMARQU\Miniconda3\envs\iris_app\lib\logging\__init__.py", line 1489, in warning
    self._log(WARNING, msg, args, **kwargs)
TypeError: Logger._log() got an unexpected keyword argument 'Name'
>>> import seqlog
>>> seqlog.log_to_console(override_root_logger=True)
>>> logging.getLogger("Example").warning("Hello", extra={"Name": "World"})  # "Name" wouldn't show up in Seq
WARNING:Example:Hello
>>> logging.getLogger("Example").warning("Hello", extra={"log_props": {"Name": "World"}})  # "Name" wouldn't show up in Seq
WARNING:Example:Hello
>>> logging.getLogger("Example").warning("Hello", Name="World")
WARNING:Example:Hello

[tintoy]

Hi.

I think this happens because we treat extra as reserved because that's the keyword argument we have to use internally to pass additional properties for submission as log event properties:

https://github.com/tintoy/seqlog/blob/573912a0ce6c5016cfbf3692ef4b0f1f697fc4fa/seqlog/structured_logging.py#L189 https://github.com/tintoy/seqlog/blob/573912a0ce6c5016cfbf3692ef4b0f1f697fc4fa/seqlog/structured_logging.py#L22

I believe you can probably do what you want by using an argument whose name is other than extra but as the logger is currently designed, anything passed in as extra does not get added to the log event properties.

I guess I could explicitly take anything that appears under extra and copy it into the log event but I'd need to put that behaviour behind a switch when configuring the logger (since there's no guarantee that other consumers are using extra the same way and I don't want to break existing consumers if possible).

Would that work for you?

[moqmar]

That would be great! Main reason why I'd like this is that libraries can then log extra things without considering whether the main application is later using Seq or not - in that case the main application has control over seqlog anyways, so a switch would work fine.

[Vacant0mens]

@tintoy Has that commit since been updated and/or fixed? or is this issue still ongoing?

[tintoy]

Thanks for the reminder! I'd completely forgot about this one 🙂

I think this change is good to go (just need to publish a new package; I was trying to find some time to set up CI to automatically publish the package which has been holding that up but I'll try to manually publish it sometime today).

[tintoy]

Published v0.3.27 🙂