python-trio / trio

Trio – a friendly Python library for async concurrency and I/O
https://trio.readthedocs.io
Other
5.98k stars 325 forks source link

Could this lib I made be added to the Awesome Libraries page? #2963

Open obnoxiousish opened 4 months ago

obnoxiousish commented 4 months ago

https://pypi.org/project/trio-mongodb/ github.com/obnoxiousish/trio-mongodb

It uses multiprocess+trio.to_sync to process pymongo queries Or just trio.to_sync in the main process alternatively

Couldn't find any other nosql dbms option using trio, there is only *SQL or redis options, would be better if it was true async but would take 5ever to implement mongodb from scratch

CoolCat467 commented 4 months ago

Pypi Project homepage link points to 404

arthur-tacca commented 4 months ago

By to_sync I guess you mean to_thread.run_sync()

Using threads only can potentially result in a massive slow down for your main process

Above is from your readme. Have you tested this? It could well be true but, then again, it might not be: Python releases the GIL during I/O so I would normally expect something I/O heavy like database client to scale very well across threads. Then again, if pymongo does a lot of processing to the data in pure Python (i.e., if it is poorly written) then it could be true. I'm just saying I wouldn't make a statement like that without some testing first.

A couple of other suggestions:

CoolCat467 commented 3 months ago

I had similar thoughts with adding support for async context managers when I took a look at the project the other day, and I was also questioning if just moving everything to another thread would be helpful. Having some sort of comparison data would be very helpful. If there were a small script for generating comparison data, I would be happy to assist with your data collection!