To the best of my knowledge, test suites supported by compiletest currently rely on external dependencies in e.g. ui or run-make test suites but we do not have any sanity checks on their versions or even the specific external dependency used. For example, which C/C++ compilers are picked for a given environment usually calls out to something like the cc crate, but we don't otherwise pin them down. This makes reproducing the tests locally very challenging or near impossible. Or for Python, I don't think we sanity check which Python versions are "supported" by the test suites.
To the best of my knowledge, test suites supported by compiletest currently rely on external dependencies in e.g. ui or run-make test suites but we do not have any sanity checks on their versions or even the specific external dependency used. For example, which C/C++ compilers are picked for a given environment usually calls out to something like the
cc
crate, but we don't otherwise pin them down. This makes reproducing the tests locally very challenging or near impossible. Or for Python, I don't think we sanity check which Python versions are "supported" by the test suites.