"Compare Receive/Verified" not Showing Diff

[BrunoJuchli]

Rider 2022.1.2 Verify Support 2022.1.0

When a test fails with a VerifyException and i click the "Compare Receive/Verified" context menu entry the diff tool is not shown. "Accept Received" works as expected.

I did try running Rider in internal mode and it doesn't show an error/exception.

I've installed Rider in the default path (i checked and it matches one of the patterns), as well as VS and VS Code. After the rider plugin didn't work as expected I've also installed DiffEngineTray. It's been a while since I read about that tool in more detail but as I remember it should automagically open the diff tool when the test fails. It does not. However, it shows the failing tests in "pending moves". Since this also doesn't work I believe this to be a problem with the DiffEngine project and not with this plugin - am I right?

If yes, then it would be nice if the error/issue would be more discoverable.

If not, is there some other rider plugin related experiments I can do or logs I can gather to find the issue?

Thank you

Note: the DiffEngineTray logs state:

2022-07-13 11:06:38.666 +02:00 [INF] MoveAdded. Target:FormatterIseTest.Should_NotHaveEmptyStream_When_DataIsAvailable.verified.bin, CanKill:false, Process:null 2022-07-13 11:06:44.809 +02:00 [INF] Did not kill for FormatterIseTest.Should_NotHaveEmptyStream_When_DataIsAvailable since CanKill=false 2022-07-13 11:07:23.433 +02:00 [INF] MoveAdded. Target:FormatterIseTest.Should_NotHaveEmptyStream_When_DataIsAvailable.verified.bin, CanKill:false, Process:null 2022-07-13 11:12:53.396 +02:00 [INF] MoveUpdated. Target:FormatterIseTest.Should_NotHaveEmptyStream_When_DataIsAvailable.verified.bin, CanKill:false, Process:null 2022-07-13 11:18:32.653 +02:00 [INF] MoveUpdated. Target:FormatterIseTest.Should_NotHaveEmptyStream_When_DataIsAvailable.verified.bin, CanKill:false, Process:null 2022-07-13 11:26:35.824 +02:00 [INF] Did not kill for FormatterIseTest.Should_NotHaveEmptyStream_When_DataIsAvailable since CanKill=false 2022-07-13 11:36:34.936 +02:00 [INF] MoveAdded. Target:FormatterIseTest.Should_NotHaveEmptyStream_When_DataIsAvailable.verified.bin, CanKill:false, Process:null 2022-07-13 11:36:58.102 +02:00 [INF] Did not kill for FormatterIseTest.Should_NotHaveEmptyStream_When_DataIsAvailable since CanKill=false 2022-07-13 17:20:22.535 +02:00 [INF] MoveAdded. Target:FormatterIseTest.Should_NotHaveEmptyStream_When_DataIsAvailable.verified.bin, CanKill:false, Process:null 2022-07-13 17:20:26.171 +02:00 [INF] MoveUpdated. Target:FormatterIseTest.Should_NotHaveEmptyStream_When_DataIsAvailable.verified.bin, CanKill:false, Process:null 2022-07-13 17:20:38.368 +02:00 [INF] Did not kill for FormatterIseTest.Should_NotHaveEmptyStream_When_DataIsAvailable since CanKill=false 2022-07-13 17:20:57.459 +02:00 [INF] MoveAdded. Target:FormatterIseTest.Should_NotHaveEmptyStream_When_DataIsAvailable.verified.bin, CanKill:false, Process:null 2022-07-13 17:21:10.039 +02:00 [INF] MoveUpdated. Target:FormatterIseTest.Should_NotHaveEmptyStream_When_DataIsAvailable.verified.bin, CanKill:false, Process:null

[SimonCropp]

what version of Verify are you using?

[BrunoJuchli]

@SimonCropp Sorry for the late answer; I was on holiday and without a computer.

Version originally used was:

Verify: 17.2.1 (nuget) Verify.NUnit: 17.2.1 DiffEngine: 9.1.0

Today I've updated Verify and Verify.NUnit to 17.5, performed git clean -xdf, reopened rider. Still the same behavior.

As this project is still using packages.config I migrated it to package reference and tried again. Still the same behavior.

[BrunoJuchli]

Today I updated to Verify: 17.10.2 Verify.NUnit: 17.10.2 DiffEngine: 10.0.0

Rider is version 2022.2.1 and the plugin Verify.Support is version 2022.2.1.

The behavior is still the same:

• "Accept Received" works
• "Compare Received/Verified": nothing happens

Tried it also with Rider in Internal mode and no exception is being logged.

[matkoch]

Maybe you could clone this repository and start debugging it. It should be a one-click experience in Rider.

Then it would be interesting if file.Received and file.Accepted are existing (or if anything else fails before).

https://github.com/matkoch/resharper-verify/blob/master/src/dotnet/ReSharperPlugin.Verify/VerifyCompareAction.cs

[BrunoJuchli]

Thanks @matkoch From looking at the code I'm deducing that DiffRunner.Launch fails. It returns a result, but unfortunately errors are silently swallowed. I created #13 to attempt to improve on the situation.

[matkoch]

Thanks Bruno, I will take a look asap. What was the error specifically?

[BrunoJuchli]

@matkoch LaunchResult.NoDiffToolFound

Note that I didn't actually get around to debug the plugin itself but by looking at the source code of the plugin and of DiffEngine I suspected that should be the issue. I verified by calling DiffEngine directly from the unit test.

This way I found that DiffEngine actually finds 3 diff tools (Rider, VS, VSCode) but the problem lies with the file extension. My unit test code looked like this:

await Verifier.Verify(memoryStream.ToArray());

This is considered binary output with .bin extension. Doesn't look like DiffEngine is considering any of the diff tools as supporting that. Since I'm actually writing an xml I "fixed" this by:

var settings = new VerifySettings();
settings.UseExtension("xml");
await Verifier.Verify(
    outputStream.ToArray(),
    settings);

Now the plugin behaves as expected.

13 will not give the user enough information to simply just fix the issue, but it will help move a step closer.

To improve upon that DiffEngine would need to use a more elaborate result model instead of just an enum. I shall consider creating a PR for that ;-)

---

Note to other readers: for xml specifically there may be a better solution (https://github.com/VerifyTests/Verify/issues/585).

[SimonCropp]

@BrunoJuchli the following should work for u

await Verifier.Verify(outputStream)
     .UseExtension("xml";