search

schireson / pytest-mock-resources

Pytest Fixtures that let you actually test against external resource (Postgres, Mongo, Redshift...) dependent code.
https://pytest-mock-resources.readthedocs.io/en/latest/quickstart.html
MIT License
179 stars 19 forks source link

get_sqlalchemy_engine() not compatible with asyncpg #182

Closed devanubis closed 1 year ago

devanubis commented 1 year ago

Describe the bug I'm unable to use pytest_mock_resources.create_postgres_fixture(async_=True) without psychopg2 installed.

I'm seeing:

E       RuntimeError: Cannot use postgres/redshift fixtures without psycopg2.
E       pip install pytest-mock-resources[postgres] or pytest-mock-resources[[postgres-binary].
E       Additionally, pip install pytest-mock-resources[redshift] for redshift fixtures.

It's not documented that pytest_mock_resources.sqlalchemy.create_async_engine() should be used, and also that doesn't auto-create the container either.

Environment

To Reproduce Steps to reproduce the behavior:

  1. pip install pytest_mock_resources[postgres-async]
  2. Run pytest_mock_resources.create_postgres_fixture(async_=True)

Additional context I'd expect either:

  1. create_postgres_fixture() to work out the postgres driver to connect with, and not always depend on psycopg2
  2. A separate fixture create_postgres_fixture_async()
  3. Bare minium the postgres-async extra should install everything needed inc psychopg2 but that's adding dev dependencies.
DanCardin commented 1 year ago

Internally, PMR is operating entirely synchronously and currently hardcodes the psycopg2 for what's probably purely historical reasons at this point. I'm investigating what it'll take to make it use sqlalchemy's native way of producing a sync engine from the async one, to remove that blocker.

Beyond that, we should probably update the documentation/error messaging a bit. I would recommend against using the driver-specific extras in general; but we can't remove them yet without potentially breaking users. The docker extra is the one installing useful dependencies, whereas the driver-specific ones are really just including drivers that you should have already chosen.

...all of that assuming we've resolved the original issue and psycopg2 is not unnecessarily required for you to run tests.

DanCardin commented 1 year ago

I have a draft implementation for this completed here https://github.com/schireson/pytest-mock-resources/pull/183. Although it's not yet passing in CI, it works for me locally, if you wanted to test it out to verify that it's going to solve your specific issue.

devanubis commented 1 year ago

Thanks I'll give it a try this afternoon!

DanCardin commented 1 year ago

Version 2.6.8 (just released) should address this bug, lmk if you agree.

devanubis commented 1 year ago

Hi @DanCardin, sorry for the delay, hectic week.

That's fixed my issue, I no longer need to install psychopg2 to get a working postgres/timescale container deployed for tests.

Thank you very much for the rapid fix :rocket: