c4-project / c4t

Runs concurrent C compiler tests
MIT License
1 stars 0 forks source link

Implement systematic perturber #123

Open MattWindsor91 opened 3 years ago

MattWindsor91 commented 3 years ago

The perturber (part of the test that takes a template plan and produces a plan with a sampled is currently entirely random.

John and Ally suggested that it would be nice to have a systematic mode whereby the tester progressively enumerates every possible perturbance of the plan.

If Go had generators, it would look something like this:

// on one machine
for _, s := range samplesOf(subjects) {
  // probably *in parallel* for each compiler, making sure to handle different compiler configuration surfaces
  for _, c := range compilers {
    for _, m := range c.marches {
      for _, o := range c.opts {
        yield()
      }
    }
  }
} 

The way I see this working is something like the following:

If we play our cards right, we might be able to make it possible to replace setc with perturb, by making the perturber able to admit a policy that just sets everything to user-specified levels.