how to get a isolated logger

[RonaldinhoL]

logger_a = logger.bind(name="a")
logger_a.remove()
logger_a.add("specific.log")

logger_b = logger.bind(name="b")

logger_a.info("Message A")
logger_b.info("Message B")

logger.info("test")

i mean to get a logger for specific.log with some special format, but when i call logger_a.remove(), it will destory logger_b and logger.

[Delgan]

Hi.

You can probably use logger_b = copy.deepcopy(logger_a).

See more in depth explanations in the documentation: Creating independent loggers with separate set of handlers.

[RonaldinhoL]

Hi.

You can probably use logger_b = copy.deepcopy(logger_a).

See more in depth explanations in the documentation: Creating independent loggers with separate set of handlers.

thank you for your help, but deepcopy need to remove all at first, this will affect other place simply use "logger", i think maybe we can add a interface to obain a empty new logger instance ...

[Delgan]

This is something I could consider.

Do you mind elaborating your use case please? What do you need that cannot be done with bind() and a filter, as shown in the documentation?

[RonaldinhoL]

yeah, in my case, i start many task(over x0000+), each one with a taskid, i want record task log to separate log, such as task1.log / task2.log ... use bind and filter maybe could result in low efficiency

[RonaldinhoL]

and use filter msg will log to global "logger" too, that is i do not want

[RonaldinhoL]

i use configed global "logger" for convenience, so do i want a free logger instance too :)

[Delgan]

Thanks. Your use case is definitely not compatible with the binder() / filter workaround. That's why there was the copy.deepcopy() solution. However, as you mentioned, this is not very convenient if the logger is already configured with non-copyable handlers, and it can raise problems with multi-threading.

Some time ago I stated that Loguru wasn't meant to be used with multiple loggers (see https://github.com/Delgan/loguru/issues/72#issuecomment-544262787). However, I've changed my mind and will probably add a logger.new() method in charge of returning a independent Logger cleaned of all handlers and configuration.

[Delgan]

In the meantime you can probably add base_logger = copy.deepcopy(logger) at the very beginning of your application and later use logger_b = copy.deepcopy(base_logger) before starting your worker. That's way, the base_logger will not contain any handler nor configuration.

[RonaldinhoL]

In the meantime you can probably add base_logger = copy.deepcopy(logger) at the very beginning of your application and later use logger_b = copy.deepcopy(base_logger) before starting your worker. That's way, the base_logger will not contain any handler nor configuration.

than i can reconfigure global logger again，is that right?

[Delgan]

Yeah, keep using logger as usual. However, when you need to create a new independent handler, do the following:

logger_a = copy.deepcopy(base_logger)
logger_a.add("specific.log")

[RonaldinhoL]

ok，thank you very much🤣

[RonaldinhoL]

is it possible to use thousands of logger instance that with independent file? I wonder if I'm using it wrong way because of max open file limit ...

[RonaldinhoL]

maybe is better to merge log to one file and record with identity like what you show

logger.add("file.log", format="{extra[ip]} {extra[user]} {message}")
context_logger = logger.bind(ip="192.168.0.1", user="someone")
context_logger.info("Contextualize your logger easily")
context_logger.bind(user="someone_else").info("Inline binding of extra attribute")
context_logger.info("Use kwargs to add context during formatting: {user}", user="anybody")

[Delgan]

I hadn't thought about it but I think you are right. It's better to limit the number of files open at the same time, because the OS can put a limit on it.

That's indeed why I prefer to use a single logger, with a small number of immutable handlers. This way, logging remains simple and limits the risk of problems.

The logs can then be post-processed if needed and each worker easily identified thanks to bind().

[kevinderuijter]

For anyone still stumbling on this issue.

I have a complex situation where I need to define a different logger for both a child and parent module. Creating a separate logger worked for me and prevented modifying the logger variable globally.

from loguru._logger import Core, Logger

logger = Logger(
    core=Core(),
    exception=None,
    depth=0,
    record=False,
    lazy=False,
    colors=True,
    raw=False,
    capture=True,
    patchers=[],
    extra={},
)

[Delgan]

@kevinderuijter Note that you're relying on a private API whose stability is not guaranteed and which may break your application in future updates.

Why not use the copy.deepcopy() workaround?

Anyway I plan to add a public logger.new() in next release.

[kevinderuijter]

@Delgan Thank you for the warning, I am aware of the implications. If the logger.new() comes through, I'll switch to that instead.

When I use deep copy, at least in my scenario, I get the following exception. E TypeError: cannot pickle '_io.TextIOWrapper' object or E TypeError: cannot pickle 'EncodedFile' object

[Delgan]

@kevinderuijter Yes, you need to remove() then re-add() the default handler at the very beginning of your application.

# Remove the default handler causing copy to fail.
logger.remove()

# Create an independent empty logger.
logger_template = copy.deepcopy(logger)

# Restore default handler.
logger.add(sys.stderr)

# Whenever you need a new independent logger, just copy the template.
new_logger = copy.deepcopy(logger_template)

Not very convenient, I agree.

[kevinderuijter]

@Delgan Somehow calling new_logger.configure(handlers=[..]) changes the handlers for logger too, whilst creating a new Logger() doesn't have that effect.

• I have a partially customised version of the documented InterceptHandler(logging.Handler)
• I use private_logger.configure(handlers=[...]) to prevent logs from the parent module from being written to stdout.

# ✅
Parent module : private_logger = Logger(...)
Child module: logger = loguru.logger

# ❌
Parent module: new_logger = copy.deepcopy(logger_template)
Child module: logger = logger.loguru

I wouldn't mind giving more context or diving deeper but I think I should open another Issue in that case. Just wanted to share a possible solution for anyone who might be struggling with this issue.

[Delgan]

@kevinderuijter Weird, that's not what I'm observing with the following code:

from loguru import logger
import sys
import copy

logger.remove()
logger_template = copy.deepcopy(logger)
new_logger = copy.deepcopy(logger_template)
new_logger.configure(handlers=[{"sink": sys.stderr}])

logger.info("Nothing printed")
logger_template.info("Nothing printed")
new_logger.info("Something printed")

Feel free to open a new issue if you want to discuss that. But I guess it's not that important since you've found another workaround. In any case, the future logger.new() should solve the problem.

[serozhenka]

@Delgan any updates on logger.new() implementation?

[Delgan]

@serozhenka Np ETA so far, sorry.

[serozhenka]

@Delgan am I able to help anyhow?

[Delgan]

Thanks for the proposal, @serozhenka.

There is a lot to be done: implementing the function, documenting it, testing it. This is no small task for an external contributor. You could give it a try, but I don't want you to invest too much time in this. In the end, it's not complicated for me to implement it, but I need to spend time on it.

You could use the copy.deepcopy(logger) in the meantime.