asyncio + futures support

[FPurchess]

+1

[klen]

+2 :)

[sdfsdhgjkbmnmxc]

+1

[jie]

+1

[msoedov]

Bump

[stleon]

+1

[pylover]

+1

[pasaranax]

+1

[jammie987]

+1

[th0mas]

With new async web frameworks such as Sanic, I would love to be able to use this library with the new async/await syntax.

+1

[ziwon]

+1

[cemoody]

+1

[nickmetal]

Any news?

[cypreess]

+100 ;-)

[pingf]

+1

[pawelswiecki]

+1

[anyUesr]

+1

[cyberbudy]

+1

[ulope]

It would be great to also support trio (an alternative event loop implementation on top of async/await with a much nicer API)

[jgirardet]

https://docs.ponyorm.com/transactions.html#using-db-session-with-generator-functions-or-coroutines is that not enough for now ?

[dekoza]

I'd love to give it a spin using trio.

[je111ena]

Any news regarding this?

[sunshineinwater]

+1 Sanic needs

[AstraLuma]

Ok, so changes that native async needs:

1. Do not perform blocking IO. That means that you need a new set of database libraries (eg asyncpg).
2. The asyncness of functions that do IO has to trickle up. So compiling SQL is a CPU-bound task and can still be done the same way. But the functions that send those queries to the database library have to be async, and the functions that glue together the user-facing APIs with the query compiler and the query executors become async, and then the actual public APIs become async.

Honestly, the simpler way is to use the integration of asyncio and concurrent.futures to make async proxies that send the work to a background thread, rather than trying to retool an entire library to be async. Preferably with a heavy dose of metaprogramming.

And no, I don't know of any tools that will allow a single source to act as both sync and async. In theory, you could strip all the async-related keywords, but you still need a way to interact with the libraries you talk to (either by switching libraries or wrapping in a sync-async bridge).

Trust me, I've been down this road, I have my own library that attempts to live in both worlds. It's not great.

[JuniorJPDJ]

@astronouth7303 telethon is using own synchronizer for async API to be able to be used as sync lib You should take a look ;)

[phacic]

+2

[MioYvo]

+1

[jinCN]

When Pony does not perform blocking IO and start use async/await, will lazy fetch still be usable?

I mean, before thinking modifying Pony. Lazy fetch does conflict with async/await. How can this problem be solved?

Ok, so changes that native async needs:

1. Do not perform blocking IO. That means that you need a new set of database libraries (eg asyncpg).
2. The asyncness of functions that do IO has to trickle up. So compiling SQL is a CPU-bound task and can still be done the same way. But the functions that send those queries to the database library have to be async, and the functions that glue together the user-facing APIs with the query compiler and the query executors become async, and then the actual public APIs become async.

Honestly, the simpler way is to use the integration of asyncio and concurrent.futures to make async proxies that send the work to a background thread, rather than trying to retool an entire library to be async. Preferably with a heavy dose of metaprogramming.

And no, I don't know of any tools that will allow a single source to act as both sync and async. In theory, you could strip all the async-related keywords, but you still need a way to interact with the libraries you talk to (either by switching libraries or wrapping in a sync-async bridge).

Trust me, I've been down this road, I have my own library that attempts to live in both worlds. It's not great.

[levrik]

I'm wondering since the most awesome part of Pony is the SQL query builder if this could be used standalone, so it the resulting queries could be used with an async client? Or someone could build an async Pony by importing the query builder but rebuilding with async in mind?

[Ed-XCF]

I'm wondering since the most awesome part of Pony is the SQL query builder if this could be used standalone, so it the resulting queries could be used with an async client? Or someone could build an async Pony by importing the query builder but rebuilding with async in mind?

maybe a pure Pony SQL query builder is a key for aio pony

[wozniakty]

👀 I want so badly to use Pony, but async is the future. Plz <3

[homus32]

+999999999

[JuniorJPDJ]

Guys, instead of writing +1 and obfuscating discussion, please just add your finger up on issue.

[KochankovID]

+

[taylor-shift]

bump

[nightstream]

Ah, this is an issue opened since 2014........

[j4hangir]

Is there any plan for this? I saw some benchmarks for pony and it already performs surprisingly well, I'd venture using asyncio it'd get even faster making it suitable for heavy-duty projects.

[KhanhhNe]

Async would be great, but actually, if you want to use the library with Trio, you can use trio.to_thread.run_sync() (docs) to run a sync function containing the database communication. In that case, it won't block the event loop, and won't need PonyORM to be rewritten also.

[v3ss0n]

But it would nullify the advantage of asyncio frameworks by spawning threads which are memory intensive and not optimal in python , compare to async which is designed for IOs.

[ShadowJonathan]

It would also not be a convenient or nice wrapper, as Pony's ORM is supposed to take advantage of a descriptive object-method model, which you can't stuff in a single wrapper.

Unless you'd wrap every single Pony interaction into a lambda/local function to spawn on a synchronous thread (and flip-flopping between sync and async on code blocks), it's just not easily possible, that's why this issue exists; to encourage pony devs to give us a high-level API for this, even if it'll spawn those same underlying operations synchronously on a thread pool.

[KhanhhNe]

But it would nullify the advantage of asyncio frameworks by spawning threads which are memory intensive and not optimal in python , compare to async which is designed for IOs.

Actually I think it wont be too bad, considering currently I have to use threads to do work & do Pony queries anyway. An asyncio interface would help integrate better & easier with asyncio based systems, and I think minor performance tradeoff is reasonable.