VerifyTests / Verify

Verify is a snapshot tool that simplifies the assertion of complex data models and documents.
MIT License
2.47k stars 138 forks source link

Snapshot failing on macOS when updating Verify.Xunit from 24.1.0 to 24.2.0 #1262

Closed kaylumah closed 1 month ago

kaylumah commented 1 month ago

Hi, let me start by saying I am a big fan of your work, and snapshot testing has helped me a lot in my projects

Describe the bug

I've Dependabot setup in a combination with Linux based GitHub Action runner to run all units tests to make sure all my packages are still working. I think I have found a possible regression specific to the macOS. I found out about it when starting to some tasks on the repo on my Mac and it started failing. I went back to a version where the test was still working (24.1.0)

I have some code that contains an example of my path /Users/maxhamulyak/ which is now being replaced with {UserProfile}. After updating the verified files to match this behaviour, the test starts failing in CI in GitHub Actions.

Did something change from 24.1.0 -> 24.2.0 that started scrubbing profile names on macOS? Is there a way to opt-out of this?

I don't know if you have a Mac available, because the repo case of my project does have pipelines enabled, but those are currently linux based

kaylumah commented 1 month ago

Note -> I think usually the replacement that is suggested here is correct. However in this case its not the userpath, but part of a blogpost that mentions that path always

SimonCropp commented 1 month ago

currently no, there is not a way to opt out of this feature. But i am happy to accept a pull request that adds it as a feature

no i dont have a mac. but i do run CI on Mac. I previously also ran it on linux, but it is temporarily disabled since i am having problems with the net9 previews on linux

kaylumah commented 1 month ago

No worries, I can probably work around it. Was this feature introduced as part of the 24.2.0 release? Or was there as fix and should it have been falling before now?

SimonCropp commented 1 month ago

Was this feature introduced as part of the 24.2.0 release

not that i know of