Tinche / aiofiles

File support for asyncio
Apache License 2.0
2.66k stars 150 forks source link

AIOFiles Take Longer Than Normal File Operation #110

Closed ghost closed 3 years ago

ghost commented 3 years ago

hi developers I have a question I'm new to the python async world and I write some code to test the power of asyncio, I create 10 files with random content, named file1.txt, file2.txt, ..., file10.txt

here is my code:

import asyncio
import aiofiles
import time

async def reader(pack, address):
    async with aiofiles.open(address) as file:
        pack.append(await file.read())

async def main():
    content = []
    await asyncio.gather(*(reader(content, f'./file{_+1}.txt') for _ in range(10)))

    return content

def core():
    content = []
    for number in range(10):
        with open(f'./file{number+1}.txt') as file:
            content.append(file.read())

    return content

if __name__ == '__main__':
    # Asynchronous
    s = time.perf_counter()
    content = asyncio.run(main())
    e = time.perf_counter()
    print(f'Take {e - s: .3f}')

    # Synchronous
    s = time.perf_counter()
    content = core()
    e = time.perf_counter()
    print(f'Take {e - s: .3f}')

and got this result:

Asynchronous: Take 0.011
Synchronous: Take 0.001

why Asynchronous code take longer than Synchronous code ? where I do it wrong ?

Tinche commented 3 years ago

You're not doing anything wrong. What aiofiles does is delegate the file reading operations to a threadpool. This approach is going to be slower than just reading the file directly. The benefit is that while the file is being read in a different thread, your application can do something else in the main thread.

A true, cross-platform way of reading files asynchronously is not available yet, I'm afraid :)