search

RazerM / sqlalchemy_aio

Asyncio strategy for SQLAlchemy.
Other
300 stars 17 forks source link

Add notice that this is not an asyncio implementation but a threadexecutor patch #6

Closed txomon closed 6 years ago

txomon commented 6 years ago

Hello, I ran into the following problem:

ERROR:sqlalchemy.pool.NullPool:Exception during reset or similar
Traceback (most recent call last):
  File "/home/javier/.local/share/virtualenvs/mos-bot-NUxJbKoW/lib/python3.6/site-packages/sqlalchemy/pool.py", line 703, in _finalize_fairy
    fairy._reset(pool)
  File "/home/javier/.local/share/virtualenvs/mos-bot-NUxJbKoW/lib/python3.6/site-packages/sqlalchemy/pool.py", line 873, in _reset
    pool._dialect.do_rollback(self)
  File "/home/javier/.local/share/virtualenvs/mos-bot-NUxJbKoW/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 457, in do_rollback
    dbapi_connection.rollback()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 140624458737408 and this is thread id 140624655287616
ERROR:sqlalchemy.pool.NullPool:Exception closing connection <sqlite3.Connection object at 0x7fe5af411490>

And after looking at the code, I find the README is misleading, as it suggests that is a full asyncio implementation. Please if possible say that this solution helps by making the sqlalchemy calls in a parallel thread.

Cheers!

RazerM commented 6 years ago

The quote below is at the very start of the documentation. I may add it to the README but I don't think it's too much to ask that people read the documentation for more information. That said, perhaps the documentation should be linked more explicitly than in the docs link.

What is this?

It\'s not an asyncio implementation of SQLAlchemy or the drivers it uses. sqlalchemy_aio lets you use SQLAlchemy by running operations in a separate thread.

If you\'re already using run_in_executor to execute SQLAlchemy tasks, sqlalchemy_aio will work well with similar performance. If performance is critical, perhaps asyncpg can help.

txomon commented 6 years ago

Yeah sorry, I use README as a quick start guide, to see what I can expect etc. I can send a pull request proposal if you want,

Cheers,

Javier

On Tue, Feb 27, 2018, 22:34 Frazer McLean notifications@github.com wrote:

The quote below is at the very start of the documentation. I may add it to the README but I don't think it's too much to ask that people read the documentation for more information. That said, perhaps the documentation should be linked more explicitly than in the [image: docs] https://sqlalchemy-aio.readthedocs.io/en/latest/index.html link.

What is this?

It's not an asyncio implementation of SQLAlchemy or the drivers it uses. sqlalchemy_aio lets you use SQLAlchemy by running operations in a separate thread.

If you're already using run_in_executor to execute SQLAlchemy tasks, sqlalchemy_aio will work well with similar performance. If performance is critical, perhaps asyncpg https://github.com/MagicStack/asyncpg can help.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/RazerM/sqlalchemy_aio/issues/6#issuecomment-369034056, or mute the thread https://github.com/notifications/unsubscribe-auth/AAN7mtXxPRC0YTLaKO5Su2xk827vu5Rdks5tZHTugaJpZM4SPGe_ .

RazerM commented 6 years ago

Regarding the error you saw with SQLite: make sure you're explicitly closing the result proxy (if it's not being exhausted, i.e. not all rows are fetched) and connection. Otherwise, SQLAlchemy will do those implicitly outside of sqlalchemy_aio's worker thread.

I can send a pull request proposal if you want,

I'll make the change myself while I add another link to the documentation. It can't hurt to clear up any confusion.

Thanks for the feedback on this.