Open pawamoy opened 2 years ago
Hi @pawamoy,
Thank you for sharing your article, looks very interesting!
I liked the solutions you have proposed, those are very challenging techniques!
From our side, we tried to keep the tests setups as simple as possible, so we used one separate database for all tests, and migrations were run only when needed, yet automatically. The only trick we used was rolling back all the transactions after each test.
@pytest.fixture(autouse=True, scope="session")
def run_migrations():
import os
print("running migrations..")
os.system("alembic upgrade head")
Tests used the same interface for database connection as a main FastAPI app, but with different DATABASE_URL
from env and force_rollback=True
parameters.
from databases import Database
ROLLBACK_TRANSATIONS = ENVIRONMENT == "testing"
database = Database(DATABASE_URL, force_rollback=ROLLBACK_TRANSACTIONS)
force_rollback=True
means every db transaction will be rolled back after disconnection.
Hi @zhanymkanov, thanks for your answer and comments! The rollback is very clever, didn't think of that! That will surely be useful to me in other projects/tests :thinking:
I'll go ahead and close the issue now :slightly_smiling_face:
I will reopen the issue so that people could find this post easily since your article could be helpful for some people writing tests
From @pawamoy
The interesting thing in the post is how each test has access to its own, unique, temporary database :)
From @zhanymkanov
one separate database for all tests .... rolling back all the transactions after each test.
This caught my attention because this year, we moved from the first approach to the second approach and it made our tests 60% faster.
Our old fixture (roughly equivalent to the link @pawamoy sent) was consuming more time than the tests themselves.
@pytest.fixture(autouse=True)
def db_connection() -> App:
engine = create_engine(DB.url, connect_args=...)
Base.metadata.create_all(engine)
yield engine.connect()
drop_everything(engine) # carefully deletes tables in the order that they're allowed to be deleted given all FKs
I'm not sure if force_rollback=True
is right for us because:
Here's exactly how we do it now
db_connection()
fixture (above) with @pytest.fixture(scope="session")
, so we still have our DB setup, but only once for the whole pytest session.
@pytest.fixture(autouse=True) def db_tx_cleanup(db_connection, mocker): transaction = db_connection.begin() session_factory = sessionmaker(autocommit=False, autoflush=False, bind=db_connection) Session = scoped_session(session_factory)
mocker.patch("backend.db.Session", new=Session) # This is the *only* place our code can get a DB session.
yield
transaction.rollback()
**One question** @zhanymkanov
Do you mean you have a permanent DB that you run tests against? I have so many questions!!
- Does each developer have their own local one? (more stuff to set up)
- Or is it accessed over the network? (then your tests are slower because of network latency)
- Why did you decide to take this approach? To make it easier to test against Postgres instead of sqlite, and avoid the postgres startup time?
**Context**
We're using Falcon in our main "monolith", but all our (newer) microservices use FastAPI
By the way, awesome repo @zhanymkanov . Thank you.
We are evaluating whether it's worth switching from Falcon + Marshmallow to FastAPI + Pydantic. For now I don't think so, but we'll see. This repo really helped me to understand several things I was not sure of for FastAPI in bigger repos.
Hi @treharne,
Thank you for sharing your experience, it's really interesting to read your team's way of doing things!
docker compose up -d
and you have your local setup :)
I recently posted this article about testing FastAPI + Omar + Alembic applications: https://pawamoy.github.io/posts/testing-fastapi-ormar-alembic-apps/
The interesting thing in the post is how each test has access to its own, unique, temporary database :)
That's it, just sharing, feel free to close or comment!