Closed sageserpent-open closed 1 year ago
The spike was very disappointing - replaced the use of timed strategies in the Americium section of the repository https://github.com/jlink/shrinking-challenge with a custom strategy factory:
static Function<CaseSupplyCycle, CasesLimitStrategy> adaptiveShrinkingAfter(
CasesLimitStrategy discoveryCasesLimitStrategy) {
// NASTY HACK: park some local state here that records the results
// from successive cycles. This is a hack because it assumes client
// code doesn't hold on to the lambda produced by this call and
// re-use it across multiple calls to `Trails.withStrategy`. It's a
// spike, after all...
final AtomicInteger emissionCountInCycle = new AtomicInteger();
final AtomicInteger starvationCountInCycle = new AtomicInteger();
return caseSupplyCycle -> {
final CasesLimitStrategy cycleSpecificCasesLimitFactory;
if (caseSupplyCycle.isInitial())
cycleSpecificCasesLimitFactory = discoveryCasesLimitStrategy;
else {
final int emissionCountInPreviousCycle =
emissionCountInCycle.get();
final int starvationCountInPreviousCycle =
starvationCountInCycle.get();
cycleSpecificCasesLimitFactory = counted(
emissionCountInPreviousCycle,
(double) starvationCountInPreviousCycle /
emissionCountInPreviousCycle);
}
emissionCountInCycle.set(0);
starvationCountInCycle.set(0);
return new CasesLimitStrategy() {
@Override
public boolean moreToDo() {
return cycleSpecificCasesLimitFactory.moreToDo();
}
@Override
public void noteRejectionOfCase() {
emissionCountInCycle.decrementAndGet();
starvationCountInCycle.incrementAndGet();
cycleSpecificCasesLimitFactory.noteRejectionOfCase();
}
@Override
public void noteEmissionOfCase() {
emissionCountInCycle.incrementAndGet();
cycleSpecificCasesLimitFactory.noteEmissionOfCase();
}
@Override
public void noteStarvation() {
starvationCountInCycle.incrementAndGet();
cycleSpecificCasesLimitFactory.noteStarvation();
}
};
};
}
This failed miserably in minimising the failing test cases for the Bound5Spec
, CouplingTest
and CalculatorSpec
examples, which prior to the change would reliably yield optimal shrinkages - the approach taken was to feed the original timed strategy to adaptiveShrinkingAfter
.
Marking this as a won't-fix...
Closing as this isn't a good idea, based on the spike outcome.
It could be useful - especially for a Scalacheck integration - to furnish statistics from the previous cycle (or maybe across all of previous cycles) to allow customisation of the
CaseLimitStrategy
for the next shrinkage cycle. This would allow adaptive strategies to be formulated, which in the case of Scalacheck would fit the notion of an initial failure discovery cycle based on Scalacheck's configuration, followed by dedicated shrinkage cycles that are tailored by previous cycles and not by the Scalacheck configuration.It is worth doing a spike with a quick-and-dirty stateful factory for
CasesLimitStrategy
- not strictly what we're supposed to pass to the likes ofTrials.withStrategy
, but it would allow some experiments to be performed to see if it's worthwhile....