trailofbits / differ

Detecting Inconsistencies in Feature or Function Evaluations of Requirements
GNU Affero General Public License v3.0
67 stars 4 forks source link

Exhaustive Parameter Combination Generation #52

Closed ameily closed 1 year ago

ameily commented 1 year ago

Variable values are currently generated in an isolated exhaustive manner where each variable iterator is exhausted but not every combination is generated. For example, consider the following three variables:

variables:
  month:
    type: int
    range:
      minimum: 1
      maximum: 12
      count: 12

  day:
    type: int
    range:
      minimum: 1
      maximum: 10
      count: 10

The current generation procedure would only generate 12 trace contexts, which exhaustes the 12 month values and the 10 day values, where two day values will be repeated.

This is fine for some situations, but, in the example of the date binary, we need collective coverage where each unique combination is exercised, which is this case would be 120 contexts.

Create a new class for generating trace contexts in a collectively exhaustive manner, producing all unique combination of values.