Closed c-p-b closed 7 months ago
Are you able to give a reproducer of the conflict that doesn't have dependencies outside of greenback, sqlalchemy, and some async library? I'm not familiar with Prefect at all, and only a little bit familiar with sqlalchemy's use of greenlets.
Hey @oremanj — I don't work at Prefect anymore but I was the one looking into this originally and am quite curious if there's a solution.
I'll see if I can find or create an example. It's possible @jakekaplan could provide one as well.
Okay it's kind of late and I don't remember all the details so forgive me if these are not correct, but I can create a few symptoms as follows:
import greenback
import sqlalchemy
import asyncio
def fails():
greenback.await_(asyncio.sleep(0)) # ERROR
# TypeError: object NoneType can't be used in 'await' expression
async def main():
await greenback.ensure_portal()
greenback.await_(asyncio.sleep(0)) # OK
await sqlalchemy._util.concurrency.greenlet_spawn(fails)
if __name__ == "__main__":
asyncio.run(main())
import greenback
import sqlalchemy
import asyncio
def fails():
sqlalchemy._util.concurrency.await_only(asyncio.sleep(0)) # OK
greenback.await_(asyncio.sleep(0)) # ERROR
# TypeError: object Value can't be used in 'await' expression
async def main():
await greenback.ensure_portal()
greenback.await_(asyncio.sleep(0)) # OK
await sqlalchemy._util.concurrency.greenlet_spawn(fails)
if __name__ == "__main__":
asyncio.run(main())
with
greenback==1.1.1
sqlalchemy==2.0.20
@oremanj just wondering if you had a chance to look into this and if I can do anything else to help. Thanks!
Hi, sorry for the delay here. It looks like sqlalchemy stores a link to the parent/driver greenlet explicitly in its child greenlet, instead of using greenlet.getcurrent().parent
. I'm guessing that would fix the issue here. I'd be happy to review a PR if you like; otherwise I'll try to get a fix up, but I'm pretty busy at the moment.
Thanks for looking into it. I tried to make some changes in that vein and must admit I'm still a bit confused. If you can provide more guidance I'm happy to play with it some more. Otherwise, I understand that you're busy.
Thanks for the library!
Prefect has been doing some experimenting with greenback here and ran into this limitation because SQLAlchemy is also used:
The workarounds for this such as running in a separate thread have their own drawbacks
Is this kind of isolation something that could be handled at the
greenback
level? Or is this more of a limitation of the way greenlets and/or SQLAlchemy works?