search

ttutisani / Xunit.Gherkin.Quick

BDD in .NET Core - using Xunit and Gherkin (compatible with both .NET Core and .NET)
MIT License
205 stars 29 forks source link

Is there a way to access @tags and if a test passed or failed from with code #60

Closed glassenbury closed 6 years ago

glassenbury commented 6 years ago

This is more a questions than issues. I've being using the Xunit.Gherkin.Quick to run UI tests in .netcore and selenium. I have got this working but have these issues:

  1. I use the tags on the scenario to perform various functions. e.g Set the browser windowsize. I need to be able to access the current scenerios tags and perform so code on start. Is this possible?
  2. At the end of scenario, can we the determine in the steps code if the test has passed or failed. I want to keep the web browser open and just navigate home if the test passed and shutdown the browser if it failed. This help performance of running these tests. (approx 1500 scenarios)
ttutisani commented 6 years ago

Hi,

I will try to answer.

  1. Accessing tags. You cannot access tags of the scenario from the step method. But maybe you can set the browser windowsize some other way? e.g. make it part of the scenario. Otherwise if you remove the tag, the scenario will fail. That actually tells me that maybe it's part of the scenario? What do you say?
  2. Determining pass/failure result. You cannot do that. Xunit does not have such thing as test context because they believe it's bad to share any information between various test executions. I could theoretically expose it but I hesitate to do so yet, again because I don't want to go against Xunit design guidelines. I'm curious though - why do you want to shut down the browser when the test fails? What's wrong with going to the home page regardless of failure or success?
glassenbury commented 6 years ago

Thanks for the quick reply.

  1. I agree this is part of the scenario. In the past we had been using the tags as parameters to pass into a scenario. I have added another step to a step base class.

  2. Determining the result of the test, I probably didn't go into enough detail. I restarted the browser on failed tests as there might be a popup in the UI or brower is mobile view and it was easier to start again with new browser instance. This does defy Xunit design guidelines to re-use the current browser over different tests but it cut's in half the execution time. The other option I have is not to dispose the browser in the steps file but at the end of the test run. Unfortunately I don't know in code when all of the tests are complete to shut the browser down. Is this possible? I have a kill any left over tasks/browsers in job outside of the test run.

In summary, we can use it as is by starting a new browser then shutting it down at the end of scenario. This is cleaner, especially when different browser sizes for different scenarios. It will just take longer but we are hoping to parallel run these in containers.

ttutisani commented 6 years ago

Okay, thanks for understanding why I follow the Xunit guidelines.

You asked how to determine that all the tests ran. You can do that using Xunit collection fixtures. Simply attach the same fixture to all feature classes. You can see an example of this here: https://github.com/xunit/samples.xunit/blob/master/CollectionFixtureExample/InsertTests.cs Point is that the fixture gets disposed when all the corresponding tests have ran already.

I would also suggest to find a way and go to home page before every test. If there is a popup, or if it's a mobile view, then you should ensure to close the popup and reset to the default non-mobile view. Whatever it takes to reset the state, you should be doing that at the beginning of every unit test. That's how I would recommend.

I hope this helps.

glassenbury commented 6 years ago

Thanks, this has been very useful.