This PR adds some basic tooling to validate benchmarks configurations added to multicore_*_run_config.json files. We also add a CONTRIBUTING.md file that gives some basic instructions on adding a parallel benchmark.
The PR also modifies the existing JSON files to fix validation errors. This validation removes the benchmarks listed in #446 from the nightly runs. The run_config.json files need to be triaged separately to see why some of the benchmarks are skipped from nightly runs (don't have the macro_bench tag), and need to be fixed.
This PR adds some basic tooling to validate benchmarks configurations added to
multicore_*_run_config.jsonfiles. We also add a CONTRIBUTING.md file that gives some basic instructions on adding a parallel benchmark.The PR also modifies the existing JSON files to fix validation errors. This validation removes the benchmarks listed in #446 from the nightly runs. The run_config.json files need to be triaged separately to see why some of the benchmarks are skipped from nightly runs (don't have the
macro_benchtag), and need to be fixed.