search

JoshKeegan / xRetry

Retry running tests via Xunit and Specflow
MIT License
54 stars 14 forks source link

Strange behaviour/retry not working #75

Closed dilandau2001 closed 3 years ago

dilandau2001 commented 3 years ago

Hi.

I was looking for the retry feature in a project and I first came out with the XRetry nuget package. I try it and it did not work. I then came to the git code and try directly with the code and I suffer the same extrange behaviour.

Visual Studio 2019. xunit 2.4.1 xunit.extensibility.Core 2.4.1 xunit.runnner.visualstudio 2.4.1 / I tried also 2.4.3 with same results. Projects are build in .Net Frawework 4.8

When I build the solution the tests are discovered either by visual studio and by resharper test explorers which sound promising from start.

But when I run the tests, those test decorated with the Retry attributes just dissapear. I have to rebuild the solution to make them appear again.

If I try to debug the test and put breakpoints inside retry code, I can see that it reaches the RetryFactAttribute.cs code but nothing else.

If I debug with all exceptions I can see multiple exceptions before test start but more or less the same exceptions than a woking test with normal xunit attributes.

I have tried xunit-retry library which have very similar code than xRetry and that one works but that one does not have theoryfact. I would prefer to make xRetry work than picking the other one and I am struggling to know what is happening.

I have tried to make my own Fact inheritor but I have the same problem and it is like the Discover is not really found or like if [XunitTestCaseDiscoverer("xRetry.RetryFactDiscoverer", "xRetry")] is not working, but the other library have similar logic and it works, so I am right now clueless.

JoshKeegan commented 3 years ago

Thankyou for the details. I have reproduced the issue and will look into it.

JoshKeegan commented 3 years ago

I have been working on a fix for this which will be being released very shortly (the build is queued now: https://gitlab.com/JoshKeegan/xRetry/-/pipelines/260612253). Can you please upgrade to v1.4.1 of the xRetry nuget package and see if it fixes your problem?

To prevent .Net Framework problems coming back again in a future release I have also updated the unit tests to run under both .Net 5.0 (as previously) but also .Net Framework 4.8, so issues with either will now break the build.

JoshKeegan commented 3 years ago

Hi @dilandau2001, I'm going to close this issue now, but please let me know if you have further issues. It'd be great if you could confirm whether it's now working for you after the latest update.