What about supporting writing files asynchronously? (hashtag630)

[vikramsubramanian]

I may open a PR implementing that.

[vikramsubramanian]

Using enqueue=True should write to file asynchronously already, right?

[vikramsubramanian]

Using enqueue=True should write to file asynchronously already, right?

enqueue=True writes files in another thread and is not a performance-best solution. There are some libs supporting native asynchronous file operations (e.g., [aiofile]( so implementing a native asynchronous file writer can potentially improve the performance of those asynchronous programs using loguru.

[vikramsubramanian]

Indeed. As Loguru supports async sink, what about the following:

from loguru import logger
import asyncio
from aiofile import AIOFile

async def main():
    logger.remove()

    async with AIOFile("/tmp/file.log", 'w+') as afp:
        logger.add(afp.write)
        logger.info("Test message")
        await logger.complete()
        await afp.fsync()

        print(await afp.read())

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

It seems to work pretty well. The downside is that it's not possible then to benefit from file features (rotation, compression, retention). Did you have something else in mind? Implementing "asynchronous file handler" natively in Loguru may require adding new dependencies, which I would prefer to avoid.

[vikramsubramanian]

The downside is that it's not possible then to benefit from file features (rotation, compression, retention).

Unasynchronizable rotation and retention are not really a problem since they are intended to not occur often. As for compression, we could write a warning to the docs to tell them that it is unasynchronizable.

Implementing "asynchronous file handler" natively in Loguru may require adding new dependencies, which I would prefer to avoid.

What about an extra_require? That's OK if you are against adding new deps. If so, just close this issue. Anyway, thanks for your replies in time. ☺️

[vikramsubramanian]

Hey. Sorry for the late answer. :grimacing:

I like the idea of an asynchronous file logger but for the moment I will not add such sink the codebase. Such handler requires new development and tests to be integrated into Loguru. It will also be necessary to publicly document its use, so adding a dependency being optional is not very adequate in this case, I think.

For now I prefer to limit Loguru to manage generic asyncio sinks. This still allows anyone to implement and use an asynchronous file handler if desired. I might reconsider my position if Python includes a built-in and easy way to do asynchronous IO, but it doesn't seem to be well handled by all systems: