pyiron / pyiron_atomistics

pyiron_atomistics - an integrated development environment (IDE) for atomistic simulation in computational materials science.
https://pyiron-atomistics.readthedocs.io
BSD 3-Clause "New" or "Revised" License
42 stars 15 forks source link

[patch] Replace conda-incubator with pyrion/actions #1506

Closed liamhuber closed 1 month ago

liamhuber commented 1 month ago

Only for the "old" tests, so we can see a speed comparison with the other unit tests

coveralls commented 1 month ago

Pull Request Test Coverage Report for Build 10099055958

Details


Totals Coverage Status
Change from base Build 10071387481: 0.0%
Covered Lines: 10701
Relevant Lines: 15091

💛 - Coveralls
liamhuber commented 1 month ago

Let's compare between conda-incubator/setup-miniconda and pyiron/actions/cached-miniconda. The comparison is imperfect because the envs will be a bit different, and it's python 3.9 vs 3.10, but the results are striking enough that I think this is just a second order effect:

So for pyiron_atomistics, using the cache costs +33% on the first invocation per-env per-day, and on every subsequent invocation costs -60%.

jan-janssen commented 1 month ago

I am not really sure what is the purpose of this. The focus of the pyiron community is to gain novel scientific insights in materials science. We develop workflow tools to address scientific challenges.

Continuous Integration is not our core expertise, so I highly recommend we stay as close to the standard as possible. From my perspective this is using the standard conda actions. This has the advantage that users with prior experience in continuous integration already know the Github actions and if they do not know they can ask ChatGPT about it or search the internet. When something goes wrong with the conda-forge actions we are affected and in the same way when something goes wrong with our added layer on top we are affected as well, the recent issue with the Python 3.12 environments is just one example of such a case.

If longer wait times of our continuous integration environment motivate us to streamline our testing environment then I think that is a good thing, because it improves our code base and in the end has the potential to enable better science. I do not think this should motivate us to develop our own Github actions. If you want to do this for educational purposes that is fine with me, but that does not motivate me to switch away from the default solution.

liamhuber commented 1 month ago

To me this argument sounds like "lets keep wasting time and resources instead of solving a technical problem, because otherwise we'd have to understand a solution to that technical problem." To me this is not a strong argument, but we can discuss in the meeting on Monday.

jan-janssen commented 1 month ago

To me this argument sounds like "lets keep wasting time and resources instead of solving a technical problem, because otherwise we'd have to understand a solution to that technical problem." To me this is not a strong argument, but we can discuss in the meeting on Monday.

The argument is compute resources vs. human time and currently we are limited by the second.

liamhuber commented 1 month ago

Some fraction of the waiting for tests to finish time also eats dev time, although the conversion rate is not 1:1, so it is not merely compute.