search

jazzband / pip-tools

A set of tools to keep your pinned Python dependencies fresh.
https://pip-tools.rtfd.io
BSD 3-Clause "New" or "Revised" License
7.69k stars 610 forks source link

Speed up tests execution #1963

Closed atugushev closed 1 year ago

atugushev commented 1 year ago

Session-scoped fixtures

The fake_dists fixture is one of the most time-consuming fixtures, so it makes sense to make it and all related fixtures session-scoped.

Finetune tests distribution over the run

Initially, tests are distributed evenly across all worker processes. But during the execution of tests, some processes may finish running tests faster than others. In these cases, the --dist=worksteal option allows these "free" workers to "steal" tests from the queue of other, still working, processes.

This is particularly useful if the run times of different tests vary greatly, as this distribution method ensures more efficient resource usage, improving the performance of testing.

Refactored slow tests

test_generate_hashes_with_split_style_annotations and test_generate_hashes_with_line_style_annotations were utilizing too many external packages. We should always prefer local packages as they are faster and pip-compile doesn't need network to compile them.

Stats

Locally, it increases speed by about 30%. Here are CI runs:

Screenshot 2023-08-10 at 21 47 56
atugushev commented 1 year ago

FTR, I'll reopen a few times to gather the CI stats.

atugushev commented 1 year ago

@theryanwalker https://github.com/jazzband/pip-tools/pull/1967 should fix failing CI.

webknjaz commented 1 year ago

@atugushev one other thing I used to do in the past, was splitting the tests across multiple CI jobs so that it adds parallelism in an extra dimension. We could consider doing the same here.

atugushev commented 1 year ago

one other thing I used to do in the past, was splitting the tests across multiple CI jobs so that it adds parallelism in an extra dimension. We could consider doing the same here.

@webknjaz, that's an interesting idea. Do you have any suggestions in mind regarding pip-tools?

webknjaz commented 1 year ago

Just a thought. It can be implemented as a pytest option in a few lines of code. But it might be not worth it, depending on the overhead of spawning more VMs. This really needs testing to see if it makes things better or worse.