search

nunit / nunit3-vs-adapter

NUnit 3.0 Visual Studio test adapter for use under VS 2012 or later
https://nunit.org
MIT License
203 stars 105 forks source link

After upgrading NUnit to Adaptor 3 and NUnit I see a wontbeused in my VsTest for filtering tests #1154

Closed aaronportier closed 6 months ago

aaronportier commented 7 months ago

I am using NUnit3Test Adaptor and NUnit 4.1.0 for an Appium 2.0 Project in an Azure Pipelines CI/CD with VsTest running the tests. I also call a runsettings file that simply gives a yml file that tells it what devices to test on for our cloud device provider. I use Categories to Filter and set the filter to use TestCategory=UserSubscription. I use Batching mode because without it I can not get re-run failed tests to run. In Batch mode filtering tests is ignored with this message IsValidServiceResponse: Received None command..Service Workflow is not active Creating run for selected test assemblies with following parameters SourceFilter: ThisWontBeUsed TestCaseFilter: TestCategory=UserSubscription

Has NUnit or the Adaptor making the above ThisWontBeUsed issue? If this is not a NuNit issue but someone knows anything else that would be very helpful!

OsirisTerje commented 7 months ago

This looks very unfamiliar. I dont know where "SourceFilter: ThisWontBeUsed" comes from.

Can you post the build yml file ?

aaronportier commented 7 months ago

The yml is just BrowserStack/appium info app,platform name,device name,platform version,parrallsperplatform:1.

On Tue, Feb 27, 2024, 10:39 AM Terje Sandstrom @.***> wrote:

This looks very unfamiliar. I dont know where "SourceFilter: ThisWontBeUsed" comes from.

Can you post the yml file ?

— Reply to this email directly, view it on GitHub https://github.com/nunit/nunit3-vs-adapter/issues/1154#issuecomment-1966842758, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABGVOYOJTRS2Q5MKRQRV5ODYVX43BAVCNFSM6AAAAABD4HYC6WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRWHA2DENZVHA . You are receiving this because you authored the thread.Message ID: @.***>

OsirisTerje commented 7 months ago

That is not a valid Azure Pipeline yml.

I guess this issue is probably on the Appium side. You will probably get more relevant help there.

aaronportier commented 7 months ago

Hmm...I know its not thats why I did not want you to see it I am using Classic the YML is like a JSON again for devices form the provider.I guess it might be VsTest however I did not upgrade it. It's not appium related. NUnit has had many issues related to VsTest and Azure re-running failed tests so it makes me think with all that history of issues this seems like a NUnit issue again. The appium part does not matter could be Selenium could be anything there is something wrong with NUnit or VsTest not the tests as they worked till the upgrade. I upgraded NUnit that's why I think NUnit might be the culprit.

OsirisTerje commented 7 months ago

Then you have probably seen the other issues around re-running, so perhaps they can give you some hints on how to work around it. There should be nothing in the adapter (and not in nunit itself) that would cause re-run to fail. The adapter process dies between the runs.

I would check with a very small repro solution, a single test or so, and then just check the parameters used there. Repro of complex solutions is very hard to diagnose, and when using the Classic Pipeline, you don't really see what is going on.

Also, which adapter versions did you update - from/to, and same with NUnit itself, from/to version ?