It is very easy to write a bad inputs.json file, and have to iterate multiple times to correct it. In the current implementation each iteration invokes cromwell that is guaranteed to fail, creating multiple log files, directory subtrees etc.
All of that guaranteed failure can be avoided if, instead of passing a bad JSON to cromwell for execution, the test command did a preliminary validation of syntax. Something as simple
as a try block around json.load(open("tests/inputs.json")) would suffice.
It is very easy to write a bad inputs.json file, and have to iterate multiple times to correct it. In the current implementation each iteration invokes cromwell that is guaranteed to fail, creating multiple log files, directory subtrees etc.
All of that guaranteed failure can be avoided if, instead of passing a bad JSON to cromwell for execution, the test command did a preliminary validation of syntax. Something as simple as a try block around
json.load(open("tests/inputs.json"))
would suffice.