toMatchDisk_TODO throws error when writing multiple snapshots to disk during parameterized tests, succeeds on follow ups.

[lewchuk-span]

I am using selfie-runner-junit5:2.0.1 to verify errors for each test case. I my parameters include a unique test case name to pass into toMatchDisk.

The summarized version:

    @ParameterizedTest(name = "rejects invalid commands: {1}")
    @MethodSource("modifyNodeItemsValidationCases")
    fun `rejects invalid commands`(
        cmd: Command,
        testcaseName: String,
    ) = runTest {
        val thrown = shouldThrow<TraitStandardResponseException> {
            validate(cmd)
        }
        thrown.response.respCase shouldBe RespCase.RESPONSE_BAD_REQUEST
        Selfie.expectSelfie(thrown.message.toString()).toMatchDisk(testcaseName)
    }

When I run this test case, the snapshot file is successfully written:

╔═ rejects invalid commands/Child node not found ═╗
ModifyItem must have a valid item case to modify
╔═ rejects invalid commands/Empty modify item ═╗
ModifyItem must have a valid item case to modify
╔═ rejects invalid commands/Empty modify node item ═╗
ModifyNodeItemWithAddition must have either modifyItem, updateExistingTraits, modifyChildElectricalConnectionPoints, or upstreamElectricalConnectionPoint
╔═ rejects invalid commands/No UtilityMeterNode when trying to edit a utility meter node ═╗
No UtilityMeterNode item found for node id 1 in topology
╔═ rejects invalid commands/Node not found ═╗
Provided node id 1 not in topology
╔═ rejects invalid commands/empty modifyDownstreamElectricalConnectionPoint ═╗
ModifyItem must have a valid item case to modify
╔═ rejects invalid commands/empty simpleElectricalConnectionPoint ═╗
ModifyItem must have a valid item case to modify
╔═ [end of file] ═╗

But the first run fails with this error:

org.opentest4j.AssertionFailedError: Snapshot was set to multiple values!
  first time: ValidatorTest$rejects invalid commands$1.invokeSuspend(Validator.kt:613)
    this time: ValidatorTest$rejects invalid commands$1.invokeSuspend(Validator.kt:613)

Once the file is created things seem to work:

• Subsequent test runs pass
• If I modify one or more of the expected value and include //selfieonce the updates are made and tests pass
• Add one or more new test case with //selfieonce

I can produce the same errors again if the file exists if I ever reuse the _TODO.

[nedtwigg]

Thanks for the report. It would be a lot easier to debug if the error message said:

Snapshot `rejects invalid commands/Node not found` was set to multiple values, 
  first value: blah
 second value: blah

I did a little browsing, and for now I can't figure out what the problem might be without a reproducer.