Open sverhoeven opened 2 years ago
We have already unit tests that validate the examples
config files; both *-test.cfg
and all *.cfg
. There is no need for the workflow builder to test it; wrong configs in the examples/
folder won't (shouldn't) pass into haddock3 main
; tests would find it during pull request.
Yes there is @joaomcteixeira, the builder works in a different state than the source code. If it writes invalid config files these will go over the pipeline, spend computational resources and then fail, remember that computational resources cost money and time.
Yes. I understand the need to validate the configuration files created by users using the workflow builder. But there is no need for the workflow builder to test the *-test.cfg
; those are already tested.
The validator that is part of the workflow builder should also find that the examples are valid. So it is more about testing whether the JS code is OK then that the examples are OK.
The workflow build validates workflows uploaded to it. We should check that the workflows at https://github.com/haddocking/haddock3/tree/main/examples are valid.
It would be nice to be able to run
And have a way run multiple (similar to https://github.com/haddocking/haddock3/blob/main/examples/run_tests.py) in a CI job like
Additionally it would be nice to compare the toml file the builder generates is comparable to the input.