hu-macsy / simexpal

Simplifying Experimental Algorithmics
https://simexpal.readthedocs.io
MIT License
17 stars 18 forks source link

fileless instances (issue #78) and instances with extra arguments #84

Closed duylethanh closed 4 years ago

duylethanh commented 4 years ago

This PR contains the following:

Below are some more details for the first two points:

The extra_args of instances are accessible via the already existing @EXTRA_ARGS@ variable, which only resolved to the extra_args of variants before. Now it resolves to the concatenation of both (possible more, because we can have more than one variant for an experiment)extra_args (from the instance and the variants).


The validation schema for instances in the experiments.yml file can now roughly be expressed as:

if 'generator' in instance:
    check syntax for generator instances
if 'extensions' in instance:
    check syntax of multiple extension instances
if 'files' in instance:
    check syntax of arbitrary file instances
if neither of the above cases:
    check syntax for local/remote

(Notice the if-if-construct instead of an if-elif-construct which is due to the fact that jsonschema 7.0 does not currently support if-elifconstructs.)


If an entity fails a oneOf construct, simexpal will now print out the errors for each subschema, e.g,:

the entity

instances:
  - repo: local
    items:
      - name: baz
        extra_args: ['some','extra_args']
      # 'name' key is missing and 'foo' is an invalid key
      - extra_args: ['some','extra_args']
        foo: bar
  - repo: locjal    # typo of local
    items:
      - instance1

produces the following output when calling a simex command:

Screen Shot 2020-11-27 at 21 57 50
duylethanh commented 4 years ago

PR updated.

avdgrinten commented 4 years ago

LGTM.