o2r-project / erc-spec

Executable Research Compendium specification and guides
https://o2r.info/erc-spec/
Creative Commons Zero v1.0 Universal
7 stars 5 forks source link

Overhaul validation procedure (conceptionally and technically) #21

Closed nuest closed 7 years ago

ghost commented 7 years ago
nuest commented 7 years ago

On our (old...) poster we use the terms "check metadata" and "check execution" (which means "compare the execution results to reference results").

Maybe we can use the terms bag validation (still never use the word validation without bag) and ERC check or check of ERC. The latter comprises a metadata check, an execution check (the inner container runs), and a result check (= comparison against reference workspace).

The word "validation" might also be too strong for us to use anyway. We can "check the result of computations in a controlled environment", but a replication study (different data/code etc.) can validate the results.

ghost commented 7 years ago

This sounds reasonable to me and seems to be handy wording.

ghost commented 7 years ago

Results of our discussion: https://github.com/o2r-project/erc-spec/issues/35