search

sageserpent-open / americium

Generation of test case data for Scala and Java, in the spirit of QuickCheck. When your test fails, it gives you a minimised failing test case and a way of reproducing the failure immediately.
MIT License
15 stars 1 forks source link

instants() needs to support ranges of valid values #45

Closed rpolton closed 2 years ago

rpolton commented 2 years ago

Similarly to integers(), longs() etc, instants() should provide an overloaded factory that accepts a lower bound and an upper bound. Instant MAX defines the maximum supported Instant, '1000000000-12-31T23:59:59.999999999Z' and this is far in the future. The issue is that the standard DateTimeFormatter renders any year larger than 9999 with a '+' prefix which is potentially unexpected. In order to avoid having to code for the year 10000 (I'd like to think that Java will have been updated to handle 5-digit years consistently by the time we approach that point) it would be better to restrict the range of possible values in the factory method. The application of a filter() after construction works sometimes but often simply produces an empty stream -> Precondition violation in Trials.

sageserpent-open commented 2 years ago

There is a TrialsApi.stream method that takes a CaseFactory that allows this to be done (mapping to Instant also works off Trials.longs) - but it would be nice to have parity between Trials.instants and Trials.integers / Trials.characters.

sageserpent-open commented 2 years ago

Released in version 1.4.6, GIT SHA: 13474b963286baa851d79771abd17a897fb40d13

sageserpent-open commented 2 years ago

Evidence made using JShell running against Americium version 1.4.6 follows...

Code:

import com.sageserpent.americium.java.Trials;
import com.sageserpent.americium.java.TrialsApi;

import java.time.Instant;

import static org.junit.Assert.assertThrows;

final TrialsApi api = Trials.api();

final Instant shrinkageTarget = Instant.now();

System.out.println("Shrinkage target inside the interval:");

assertThrows(RuntimeException.class, () -> api.instants(shrinkageTarget.minusSeconds(60 * 100), shrinkageTarget.plusSeconds(60 * 1000), shrinkageTarget).withLimit(10).supplyTo(caze -> {
    System.out.println(caze);
    throw new RuntimeException();
}));

System.out.println("Shrinkage target at high end of interval:");

assertThrows(RuntimeException.class, () -> api.instants(shrinkageTarget.minusSeconds(60 * 100), shrinkageTarget, shrinkageTarget).withLimit(10).supplyTo(caze -> {
    System.out.println(caze);
    throw new RuntimeException();
}));

System.out.println("Shrinkage target at low end of interval:");

assertThrows(RuntimeException.class, () -> api.instants(shrinkageTarget, shrinkageTarget.plusSeconds(60 * 1000), shrinkageTarget).withLimit(10).supplyTo(caze -> {
    System.out.println(caze);
    throw new RuntimeException();
}));

System.out.println("Shrinkage target is implicitly defined at the epoch.");

assertThrows(RuntimeException.class, () -> api.instants(Instant.EPOCH.minusSeconds(2), Instant.EPOCH.plusSeconds(1)).withLimit(10).supplyTo(caze -> {
    System.out.println(caze);
    throw new RuntimeException();
}));

Output:


import com.sageserpent.americium.java.Trials
import com.sageserpent.americium.java.TrialsApi
import java.time.Instant
static import org.junit.Assert.assertThrows

field TrialsApi api = com.sageserpent.americium.TrialsApis$$anon$1@5f8ed237

field Instant shrinkageTarget = 2022-06-19T12:05:39.942376Z

System.out.println("Shrinkage target inside the interval:")
Shrinkage target inside the interval:

assertThrows(RuntimeException.class, () -> api.instants(shrinkageTarget.minusSeconds(60 * 100), shrinkageTarget.plusSeconds(60 * 1000), shrinkageTarget).withLimit(10).supplyTo(caze -> {
    System.out.println(caze);
    throw new RuntimeException();
})) = Trial exception with underlying cause:
java.lang.RuntimeException
Case:
2022-06-19T12:05:39.942Z
Reproduce via Java property:
trials.recipeHash=73b6b673871eb35b932019ab78b9262d
Reproduce via `withLimits` using recipe:
[
    {
        "FactoryInputOf" : {
            "input" : 1655640339942
        }
    }
]
2022-06-19T22:59:59.175Z
2022-06-19T11:14:52.731Z
2022-06-19T11:46:14.415Z
2022-06-19T12:01:57.632Z
2022-06-19T12:04:29.799Z
2022-06-19T12:04:31.662Z
2022-06-19T12:05:09.203Z
2022-06-19T12:05:18.963Z
2022-06-19T12:05:37.356Z
2022-06-19T12:05:37.676Z
2022-06-19T12:05:38.582Z
2022-06-19T12:05:39.013Z
2022-06-19T12:05:39.772Z
2022-06-19T12:05:39.934Z
2022-06-19T12:05:39.941Z
2022-06-19T12:05:39.942Z

System.out.println("Shrinkage target at high end of interval:")
Shrinkage target at high end of interval:

assertThrows(RuntimeException.class, () -> api.instants(shrinkageTarget.minusSeconds(60 * 100), shrinkageTarget, shrinkageTarget).withLimit(10).supplyTo(caze -> {
    System.out.println(caze);
    throw new RuntimeException();
})) = Trial exception with underlying cause:
java.lang.RuntimeException
Case:
2022-06-19T12:05:39.942Z
Reproduce via Java property:
trials.recipeHash=73b6b673871eb35b932019ab78b9262d
Reproduce via `withLimits` using recipe:
[
    {
        "FactoryInputOf" : {
            "input" : 1655640339942
        }
    }
]
2022-06-19T11:34:14.418Z
2022-06-19T12:05:12.870Z
2022-06-19T12:05:34.264Z
2022-06-19T12:05:38.257Z
2022-06-19T12:05:39.375Z
2022-06-19T12:05:39.676Z
2022-06-19T12:05:39.752Z
2022-06-19T12:05:39.840Z
2022-06-19T12:05:39.877Z
2022-06-19T12:05:39.909Z
2022-06-19T12:05:39.936Z
2022-06-19T12:05:39.937Z
2022-06-19T12:05:39.941Z
2022-06-19T12:05:39.942Z

System.out.println("Shrinkage target at low end of interval:")
Shrinkage target at low end of interval:

assertThrows(RuntimeException.class, () -> api.instants(shrinkageTarget, shrinkageTarget.plusSeconds(60 * 1000), shrinkageTarget).withLimit(10).supplyTo(caze -> {
    System.out.println(caze);
    throw new RuntimeException();
})) = Trial exception with underlying cause:
java.lang.RuntimeException
Case:
2022-06-19T12:05:39.942Z
Reproduce via Java property:
trials.recipeHash=73b6b673871eb35b932019ab78b9262d
Reproduce via `withLimits` using recipe:
[
    {
        "FactoryInputOf" : {
            "input" : 1655640339942
        }
    }
]
2022-06-19T23:31:24.699Z
2022-06-19T12:50:24.295Z
2022-06-19T12:45:09.018Z
2022-06-19T12:17:58.667Z
2022-06-19T12:12:50.178Z
2022-06-19T12:08:32.126Z
2022-06-19T12:07:12.415Z
2022-06-19T12:06:47.053Z
2022-06-19T12:06:27.638Z
2022-06-19T12:06:07.044Z
2022-06-19T12:05:48.878Z
2022-06-19T12:05:47.060Z
2022-06-19T12:05:40.269Z
2022-06-19T12:05:39.984Z
2022-06-19T12:05:39.961Z
2022-06-19T12:05:39.949Z
2022-06-19T12:05:39.945Z
2022-06-19T12:05:39.942Z

System.out.println("Shrinkage target is implicitly defined at the epoch.")
Shrinkage target is implicitly defined at the epoch.

assertThrows(RuntimeException.class, () -> api.instants(Instant.EPOCH.minusSeconds(2), Instant.EPOCH.plusSeconds(1)).withLimit(10).supplyTo(caze -> {
    System.out.println(caze);
    throw new RuntimeException();
})) = Trial exception with underlying cause:
java.lang.RuntimeException
Case:
1970-01-01T00:00:00Z
Reproduce via Java property:
trials.recipeHash=ebab32843e515600e60954a197da9391
Reproduce via `withLimits` using recipe:
[
    {
        "FactoryInputOf" : {
            "input" : 0
        }
    }
]
1970-01-01T00:00:00.057Z
1969-12-31T23:59:59.945Z
1969-12-31T23:59:59.951Z
1969-12-31T23:59:59.997Z
1969-12-31T23:59:59.999Z
1970-01-01T00:00:00Z