search

nunit / nunit3-vs-adapter

NUnit 3.0 Visual Studio test adapter for use under VS 2012 or later
https://nunit.org
MIT License
203 stars 105 forks source link

Exposing vsts properties via testcontext in case of test plan scenario. #537

Open abhishkk opened 6 years ago

abhishkk commented 6 years ago

Description: In vsts, there is vstest v2 task which is used to run tests from build or release pipeline.

From this vstest v2 task, we can run test plan scenario. A test method needs to be associated with a test case in order for it to run via test plan flow. Check here for more info on test method association.

Test method association in VS IDE is currently enabled only for MSTest v1 adapter. We (Microsoft automated testing team) are enabling the association support for MSTest, NUnit and xUnit adapters in VS 2019. Check ask from users to enable the association support here.

Once the association is enabled, users can associate test methods to test case and then run its via test plan flow of vstest v2 task in vsts pipeline.

Issue: Testers might need vsts properties like test run id, test case id, test plan id etc in test method itself. These properties are passed from vsts to test platform in TestCase object's properties bag. Test platform passes these properties further to each adapter.

We recently exposed these vsts properties to test method via testcontext for MSTest v2 adapter. Check PR here.

Ask: NUnit adapter to expose vsts properties via test context.

abhishkk commented 6 years ago

@cltshivash @PBoraMSFT : FYI

PBoraMSFT commented 6 years ago

@OsirisTerje - customer reported issue - https://github.com/Microsoft/vsts-tasks/issues/7829

shack05 commented 3 years ago

Hi, I thought I would have a think about implementing this feature since I'd find it particularly helpful.

Given that some vsts properties are specific to individual tests (TestCaseId, TestPointId, ...) I imagine the expectation would be to flow the vsts properties through to the property bag for each NUnit test.

However I can't see a way to pass additional information to tests. From what I can understand by looking at the ITestRunner and NUnitEngineAdapter types, the test adapter uses the NUnitEngineAdapter to invoke running tests, providing an NUnit TestFilter and a listener as inputs, neither of which are suitable for those needs. It seems all test specific information (properties/traits) need to be a part of the test package itself. I tried adding additional properties to the discovered tests nodes in the xml and, even though they were visible in Runner.LoadResult, the additional properties weren't accessible from the property bag in the NUnit test.

I'm happy to keep having a look but I feel I'm at a dead end, am I missing anything?