edgedb / imdbench

IMDBench — Realistic ORM benchmarking
https://edgedb.github.io/imdbench
Apache License 2.0
241 stars 24 forks source link

Fairer comparison with SqlAlchemy #67

Open jsimonlane opened 12 months ago

jsimonlane commented 12 months ago

It looks like the SqlAlchemy uses their own native async engine.

        engine = sa_asyncio.create_async_engine(
            f"postgresql+asyncpg://sqlalch_bench:edgedbbenchmark@"
            f"{ctx.db_host}:{ctx.pg_port}/sqlalch_bench"
        )

If you subbed in your (ie MagicStack's) AsyncPG library instead instead (should be a couple line change), I'd be very curious to see a comparison... that would be a revealing benchmark to see how your EdgeDB Query Optimizer is doing (vs how much of the performance difference is better serialization).

raddevon commented 11 months ago

Would this make sense as either an additional benchmark or even a replacement for the existing SqlAlchemy benchmark, @fantix? If either, I suspect maybe the former would be better since most people would probably use the SqlAlchemy native async engine.