Add support for SQLAlchemy Session getter

[dargueta-work]

I'm having a frustrating issue with testing code. Our testing system at work uses SQLAlchemy sessions in our unit tests, but because of the way fixtures work and limits in Travis, we can't use a global session in the modules where we declare our factories. It'd be great if we could declare a Meta option that would allow passing in a callable object that would return a SQLAlchemy session instead of passing in a session directly.

For example:

# session_getter.py
def get_session():
    # Do complex stuff
    return session

# model_factories.py
from my_project import session_getter

class MyFactory(factory.Factory):
    class Meta:
        model = models.MyModel
        sqlalchemy_session_factory = session_getter.get_session

At runtime instead of using _meta.session, the factory would use _meta.sqlalchemy_session_factory().

I've created a patch (see attached) as a proof of concept. No tests, as I don't want to spend too much time on this in case it gets shot down.

Thoughts? sqlalchemy_factory.txt

[rbarrois]

The idea looks great; we might also allow the sqlalchemy_session to be a callable and call it directly.

I'm interested in @jeffwidman advice on this, as he is much more knowledgeable about SQLAlchemy than me ;)

[jeffwidman]

because of the way fixtures work and limits in Travis, we can't use a global session in the modules where we declare our factories

Can you expand on this? Are you attempting to use a normal session or a scoped_session globally? The latter is thread-safe and typically the better option for most scenarios.

[dargueta-work]

Are you attempting to use a normal session or a scoped_session globally?

Trust me, scoped_session was the first thing I tried and it failed miserably. Our unit tests use regular per-class sessions and I can't change that. We're currently using testing.postgresql, dunno if that helps.

This is what we currently have:

# conftest.py
@pytest.fixture(scope='class')
def shared_postgresql(database: str, request):
    request.cls.postgresql_url = database

    engine = sa.create_engine(database)
    request.cls.engine = engine
    request.addfinalizer(engine.dispose)

# test_bases.py:
@pytest.mark.usefixtures('shared_postgresql')
class BaseDatabaseTestCase(unittest.TestCase):
    """Base class for all unit tests using the shared database."""
    engine = None
    postgresql_url = None

    # Other stuff

[stephane]

I've just pushed a small Flask project to illustrate how to use scoped sessions to speed up tests (it avoids creating/dropping of tables between each tests):

https://github.com/stephane/flask-fast-sql-tests

I also failed to provide the scoped_session to _create() method so to avoid this error:

@classmethod
    def _create(cls, model_class, *args, **kwargs):
        """Create an instance of the model, and save it to the database."""
        session = cls._meta.sqlalchemy_session
        session_persistence = cls._meta.sqlalchemy_session_persistence
        if cls._meta.force_flush:
            session_persistence = SESSION_PERSISTENCE_FLUSH

        obj = model_class(*args, **kwargs)
>       session.add(obj)
E       AttributeError: 'NoneType' object has no attribute 'add'

I overrided factory.alchemy.SQLAlchemyModelFactory with: https://github.com/stephane/flask-fast-sql-tests/blob/master/beehive/factories.py#L6

so in my tests, I need to add the instances to the session myself (https://github.com/stephane/flask-fast-sql-tests/blob/master/tests/test_hives.py#L16).

[ilansh]

I think this feature could be useful for me as well.

I have several DB clients, each instance has a session_factory property. Right now, this hack seems to be working for me:

db_client = DBClient()
factories.UserFactory._meta.sqlalchemy_session = db_client.session_factory

Does it make sense to set the sqlalchemy session dynamically as above?

[dargueta-work]

@stephane since I posted this we've developed an internal thing we're about to open source that does something similar in a pytest environment. I can try this again and see if it'll work.