search

rust-lang / rust

Empowering everyone to build reliable and efficient software.
https://www.rust-lang.org
Other
96.63k stars 12.49k forks source link

compiletest: pin or restrict supported external dependency versions #128392

Open jieyouxu opened 1 month ago

jieyouxu commented 1 month ago

To the best of my knowledge, test suites supported by compiletest currently rely on external dependencies in e.g. ui or run-make test suites but we do not have any sanity checks on their versions or even the specific external dependency used. For example, which C/C++ compilers are picked for a given environment usually calls out to something like the cc crate, but we don't otherwise pin them down. This makes reproducing the tests locally very challenging or near impossible. Or for Python, I don't think we sanity check which Python versions are "supported" by the test suites.

jieyouxu commented 1 month ago

This behavior is also replicated by the run-make-support library which obviously isn't ideal either.