search

VerifyTests / DiffEngine

Manages launching and cleanup of diff tools
MIT License
181 stars 28 forks source link

With Rider, when multiple tests fail, only two windows are shown (sequentially) and other tests permanently fail until IDE restart #484

Closed cmeeren closed 2 weeks ago

cmeeren commented 3 months ago

Minimal repro solution: TestProject.zip

  1. Open a single instance of Rider with the repro solution (do not open other solutions in other windows, to minimize confounders)
  2. Run all tests
  3. One diff window opens. Accept the changes and close.
  4. Another diff window opens. Accept the changes and close.
  5. Bug: Observe that no other diff windows open.
  6. Re-run all the tests
  7. Bug: Observe that two of the tests fail, but no diff windows open.
  8. Close Rider
  9. Bug: Observe that Rider immediately restarts, with the splash screen, and then a diff window shows. Accept the changes and close. It is possible this then repeats for the last "pending" diff window.

Using Rider 2024.1.2.

SimonCropp commented 2 months ago

can u try setting solution opening to "new window"

image

SimonCropp commented 2 months ago

if that doesnt work, i think u need to rais it with the jetbrains peeps.

cmeeren commented 2 months ago

Are you sure "Reopen projects on startup" is relevant here? Only the last bug I described has anything to do with IDE restarts. The bugs described in steps 5 and 7 do not have anything to do with the IDE restarting.

SimonCropp commented 2 months ago

for me step 3 does not repro

One diff window opens. Accept the changes and close.

when i run all tests, all diff open.

Are you sure "Reopen projects on startup" is relevant here

no. i am guessing

given i cant repro, i will need you to debug into it to work out what is wrong

SimonCropp commented 2 months ago

or, alternatively, pick a different diff tool. My preference is Beyond Compare

SimonCropp commented 2 months ago

also, a long shot, but i did fix a on process launching in 15.4.4. can u try that version?

cmeeren commented 2 months ago

also, a long shot, but i did fix a on process launching in 15.4.4. can u try that version?

Not sure how. I use this through the Verify Support plugin. So I guess that needs to be updated? Currently on version 2024.1.2 of that plugin, and I don't see an option of updating.

SimonCropp commented 2 months ago

I use this through the Verify Support plugin.

the launching of diffs during a test run is not handled by the plugin. you can get diffengine 15.4.4 by updating to Verify.Xunit 25.0.4

cmeeren commented 2 months ago

Thank you for the clarification. I tried updating, but there was no change in behavior. I also see that the setting "New window" was already selected.

SimonCropp commented 2 months ago

given DiffEngine is just calling the rider command line i dont think there is much more i can do hear. I suggest you raise it with JetBrains

SimonCropp commented 2 months ago

perhaps @khalidabuhakmeh or @matkoch can help us

image

khalidabuhakmeh commented 2 months ago

Let me bubble this up to the right folks.

cmeeren commented 2 weeks ago

This seems to have been fixed in Rider 2024.2 (at least 2024.2.1). Thanks a lot! 😁