search

conda / conda

A system-level, binary package and environment manager running on all major operating systems and platforms.
https://docs.conda.io/projects/conda/
Other
6.41k stars 1.66k forks source link

Make conda's tests 10% faster #11994

Open dholth opened 2 years ago

dholth commented 2 years ago

Checklist

What happened?

On a successful conda test run like this one https://github.com/conda/conda/actions/runs/3265018387/usage, the CI tests (not other tasks like building images or checking the CLA) take 15h 2m 10s running several times across multiple platforms.

(Since they run in parallel, we don't have to wait quite that long).

Make the tests 5% faster repeatedly.

95% 14.25h 90% 13.5h 85% 12.h

travishathaway commented 2 years ago

@dholth, so what should happen here? 😅 It's not clear to me...

dholth commented 2 years ago

@travishathaway for example, figure out which conda test is the slowest, make it faster. Or remove redundant and extra tests. Update those tests. If the test run takes < 13.5h as reported by github actions' summary then this issue is complete.

travishathaway commented 1 year ago

Okay, that sounds like that is going to be quite a bit of work! No idea when/if this will ever get prioritized... 😭😭😭

Although, this could potentially be a nice "hackathon" project if we ever manage to get a group of people together in real life to work on such a thing. There is hope!

travishathaway commented 1 year ago

I went ahead and assigned you to this issue so you can help advocate for it 😉

dholth commented 1 year ago

Comparing the ci.yml runs from around a year ago https://github.com/conda/conda/actions/workflows/tests.yml?query=created%3A%3C2023-10-18+is%3Acompleted

And the tests.yml runs from around now https://github.com/conda/conda/actions/workflows/tests.yml?query=created%3A%3C2023-10-18+is%3Acompleted

Counting the newer tests.yml runs that took 1h+ and not the fast "creates canary build only" runs,

It looks like we are taking 138% as long this year so the tests have gotten slower.

We are also running more tests in compute-time terms but we are more concerned about real-time. In my opinion they feel more consistent lately? but I don't see that in the github logs.

github-actions[bot] commented 4 days ago

Hi there, thank you for your contribution!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed automatically if no further activity occurs.

If you would like this issue to remain open please:

  1. Verify that you can still reproduce the issue at hand
  2. Comment that the issue is still reproducible and include:
    • What OS and version you reproduced the issue on
    • What steps you followed to reproduce the issue

NOTE: If this issue was closed prematurely, please leave a comment.

Thanks!