omnilib / aiosqlite

asyncio bridge to the standard sqlite3 module
https://aiosqlite.omnilib.dev
MIT License
1.16k stars 93 forks source link

pure aiosqlite3, replace threading on async tasks, resolve deadlock #297

Closed ArtemIsmagilov closed 3 months ago

ArtemIsmagilov commented 4 months ago

Description

I believe that this library would be ideal if it worked exclusively on async tasks and not on threads. So far I don’t have the competence to change what you have. you create a thread for each connection, which already contradicts the asynchronous paradigm for python. The documentation says that asynchronous code only works on tasks. By creating threads you deprive the developer of flexibility and scalability. the developer himself needs to choose how to work - create a separate thread, create a pool of connections, or work in a basic asynchronous style. I agree, the proposal is not trivial, but we need to strive for it. A typical example is aiofiles, which works on threads, which seems to me to confuse developers who were expecting one thread in action that was not blocked in action. however, there is aiofile - it actually works on tasks. a smooth transition can be observed in niquests - fork from requests. the asynchronous interface spawned threads. This moment was rewritten in the latest version. at least that's what they wrote. Of course, you can write - “what do you want from sqlite3? This is not the database for which all this is worth doing.” However, any optimization will be better for customers than none.

Details

ArtemIsmagilov commented 3 months ago

I did a little research and this is what I found out - at the moment sqlite3 itself does not support the asynchronous API (https://www.sqlite.org/asyncvfs.html). Here's an example of an asynchronous implementation in Rust, under the hood multithreading is used (https://github.com/ryanfowler/async-sqlite) . So far my complaint is not with the developer of this library. It seems that asynchronous things work much slower when working with local objects - files, unix sockets, and so on (it is clear that there is minimal latency and long task switching in the event loop). https://stackoverflow.com/questions/39803746/peewee-and-peewee-async-why-is-async-slower. So I'm closing this issue