Wrapper for asyncio - Githubissues

Hello,

Thank you for your great library and I'm looking forward to new Cython version.

I'm interested in io_uring, and now I've been writing a asyncio wrapper for this library and file operation API. https://github.com/tree4096/uring_file (forked from https://github.com/qweeze/uring_file)

Can I contribute my code on your library or community of io_uring developers?

After implemeting some optimization, I achieved relatively better performance than other async file I/O libs such as aiofiles, aiofile, and anyio. The results of performance test as below.

Test code

https://github.com/tree4096/uring_file/blob/master/performance_test.py

Writing Test

Library	Time [s]	Process Time [s]
anyio	10.69	15.62
aiofiles	4.47	6.19
uring_file :star:	1.92	1.88
aiofile*	7.51	15.08

Reading Test

Library	Time [s]	Process Time [s]
anyio	0.482	0.816
aiofiles	0.349	0.623
uring_file :star:	0.118	0.118
aiofile*	0.473	0.828

*aiofile has Linux AIO mode, but does not work on Ubuntu due to the bug. So it run on C threading mode.

Environment

OS: Ubuntu 20.04.4 LTS on WSL2 Kernel ver.: 5.10.102.1-microsoft-standard-WSL2 CPU: AMD Ryzen 7 5800U SSD: MTFDHBA1T0TDV-1AZ1AABHA Python ver.: 3.10.5

My library provides simple and efficient APIs to access the io_uring like a database session and the file I/O APIs are written over them. Here is the sample code.

# Create new Uring
uring = uring_file.Uring(sq_size=8, cq_size=64, session_sq_size=4)

# Get SQE and submit (Open file)
async with uring.session() as session:
    sqe = session.get_sqe()
    sqe.prep_openat(liburing.AT_FDCWD, b"hello.txt", os.O_RDONLY, 0o644)

# Get result of CQE
fd = sqe.result()
print("FD:", fd)

# Close file
async with uring.session() as session:
    # Raw SQE object can be accessed from the _sqe property
    liburing.io_uring_prep_close(session.get_sqe()._sqe, fd)
    # Same expression as session.get_sqe().prep_close(fd)

# Get default Uring
default_uring = uring_file.get_default_uring()

The SQEs of the sessions are automatically merged and submitted together under high frequency access, and I call it "auto bulk submit". The auto bulk submit is realized by controlling the task scheduling of asyncio in a little hacky way and it makes large performance improvement. And the overflow protection for the CQ is also implemented by using Semaphore.

I'm not sure because I'm just a beginner but I think these APIs may be useful for developing the io_uring applications on Python.

If you are interested in my library, could you give me some comments or advice?

YoSTEALTH / Liburing

Wrapper for asyncio #17

Test code

Writing Test

Reading Test

Environment