python-gino / gino

GINO Is Not ORM - a Python asyncio ORM on SQLAlchemy core.
https://python-gino.org/
Other
2.67k stars 150 forks source link

Optimize model loader do load method #786

Closed Reskov closed 3 years ago

Reskov commented 3 years ago
Reskov commented 3 years ago

It is still pretty slow, but results are better on my laptop

@contextmanager
def timer():
    real_time = time.time()
    cpu_time = time.process_time()
    yield
    real_time = time.time() - real_time
    cpu_time = time.process_time() - cpu_time

    print(f"CPU time: {cpu_time:.2f} s, Real time: {real_time:.2f} s")

async def test_model_alternative(user, event_loop):
    row = await User.query.gino.load(lambda r, c: r).first()

    # async def new_load():
    loader = ModelLoader(User)
    ctx = {}

    with timer():
        print("Full model")
        timeit.timeit(lambda: loader.do_load(row, ctx), number=20000)

    loader = ModelLoader(User, User.nickname)
    with timer():
        print("Single column loader")
        timeit.timeit(lambda: loader.do_load(row, ctx), number=20000)

    row = await db.select([User.nickname]).gino.load(lambda r, c: r).first()

    loader = ModelLoader(User)
    with timer():
        print("Partial loader")
        timeit.timeit(lambda: loader.do_load(row, ctx), number=20000)

Before:

Full model
CPU time: 1.01 s, Real time: 1.05 s
Single column loader
CPU time: 0.47 s, Real time: 0.49 s
Partial loader
CPU time: 0.74 s, Real time: 0.75 s

After:

Full model
CPU time: 0.60 s, Real time: 0.60 s
Single column loader
CPU time: 0.37 s, Real time: 0.37 s
Partial loader
CPU time: 0.49 s, Real time: 0.49 s
wwwjfy commented 3 years ago

Looks good, thanks!