Question: Is there a quick way to disable the `text` attribute when using structured logging

[matroscoe]

I can see why this might be useful but for our specific application it just means we are transmitting the same thing twice in the same message. It would be useful to not include the "text" field as it doesn't really impact us how the message looked to the user we are only concerned about the information hitting the collector (in our case datadog).

[mrkmcnamee]

You could create a custom sink to remove it.

from loguru import logger
import json
import sys

def datadog_sink(message):
    message_json = json.loads(message)
    message_json.pop("text")
    serialized = json.dumps(message_json, default=str)
    print(serialized, file=sys.stdout, flush=True)

logger.configure(handlers=[{"sink": datadog_sink, "serialize": True}])

[matroscoe]

@mrkmcnamee that works but I am curious if I can use that to remove other parts of the message such as the icon.

I am also having another issue where when I want to add context everything ends up in the extras dictionary but a tool like data dog is expecting that to be present in the root level. Is there a way to configure loguru to store those added through a patch method to the root object and not the extra dictionary?

[mrkmcnamee]

I am also using Datadog. Loguru emits a JSON structure that contains a "record" element containing all the log details. The fields in the record can be rearranged using a patcher as you mention, but to put things outside the record element you have to manipulate the message after it has been serialised, and this can only be done in the sink function. For example, this copies the level name to outside the record so Datadog can find it:

from loguru import logger
import json
import sys

def datadog_sink(message):
    message_json = json.loads(message)
    message_json["levelname"] = message_json["record"]["level"]["name"]
    serialized = json.dumps(message_json, default=str)
    print(serialized, file=sys.stdout, flush=True)

Alternatively, you could also configure the Datadog Python Grok parsers to parse the Loguru log structure.

[Delgan]

Hi.

As advised by @mrkmcnamee, you could implement your own serialization mechanism using a custom patch or format function: Serializing log messages using a custom function.

You can also use format=lambda _: "", serialize=True and text will be empty.

However, I'd like to work on improving the default serialized format and make it more convenient for log aggregation tools. The text is indeed a bit redundant, but this is to prevent the sink's format attribute from becoming meaningless. I agree that some fields such as level.icon are not very useful and could be removed. I'll also flatten much of the JSON probably. However, I won't probably merge extra to the root level by default because I'm afraid it could inadvertently override (or be overridden by) the built-in fields.

[amustaque97]

@Delgan any plans to pick this up anytime soon? I'm happy to work on the issue and share the proposed changes. My basic proposal would be to make both the properties configurable whether the dev wants to show text or not if serialize=True and extra to be shown at the root level or not.

I would appreciate your input if any. Thanks!

[Delgan]

Hi @amustaque97.

Thanks for your offer and sorry for the delayed response.

It's a subject I've kept in mind, but I haven't managed to come up with an API I like.

I believe the text attribute should be removed entirely, and the JSON structure should be flattened unconditionally (assuming this is the most common practice).

I don't think we should have additional options for that. Instead, the users should be able to create a custom procedure and adapt the output as they prefer. Just like it's possible currently with the format. This is the most generic solution, providing sane default without complicating the API with new parameters.

The only thing that bothers me is that by removing text, the format argument of logger.add() becomes useless. We may states in the docs that format is ignored if serialize=True but I don't like that. I don't like an API that let configure an argument which is actually ineffective. :confused:

I'm thinking when serialize=True, we should maybe add a new serialized parameter to the log record. Then, this could be converted to string with logger.add(sink, format="{serialized}") or directly accessed within a formatter function for advanced users who want to adjust the structure.

But all this are just ideas. As I said, I haven't yet decided which API to implement.

[didlawowo]

should be a good idea to simply get a argument format which can take json or text @Delgan could you provide some documentation for this ?