etc, just as Aqua allows configuration in Aqua.test_all(Package; settings...). Perhaps even a config toml could make sense at some point. I could see also needing to add manual overrides like "I manually verified package X, so let it through even though the automated test fails since it doesn't know the docs are hosted in another repo".
The motivation is automated verification of software release requirements and to fail CI if a dependency is added that does not meet these requirements.
I think the simplest implementation is to test the test-time Manifest.toml using #71, so that all recursive deps are covered and we have particular versions to resolve.
Suggested by my colleague @hannahilea
We could have an Aqua.jl style helper like
which one can put in their
runtests.jlto test the Manifest there against constraints like:These could be configured to some extent like
etc, just as Aqua allows configuration in
Aqua.test_all(Package; settings...). Perhaps even a config toml could make sense at some point. I could see also needing to add manual overrides like "I manually verified package X, so let it through even though the automated test fails since it doesn't know the docs are hosted in another repo".The motivation is automated verification of software release requirements and to fail CI if a dependency is added that does not meet these requirements.
I think the simplest implementation is to test the test-time Manifest.toml using #71, so that all recursive deps are covered and we have particular versions to resolve.