Open shirishnagarnz opened 5 years ago
Can I please have a reply on this issue?
@shirishnagarnz : Apologies for the delay. As of now, we don't have a way to soft assert in MSTest.
This is a new ask, so I will mark this as enhancement. We don't have plans to bring this feature in in near future, but we will certainly pick it up if we have enough votes. Thank you for you feedback!
So any solution apart from soft assert?
I want all steps to get execute even though some in-between steps get fail.
@shirishnagarnz : Unfortunately there is no way to do so as of now.
Thanks Jaya,
Don't know how many other users needs this enhancement but if it's not a big change, can your team please work on this enhancement?
Tagging @cltshivash @PBoraMSFT to look into the ask.
Hi @jayaranigarg - any connecting to DB through selenium is available in MSTEST or MSTESTV2?
Please share some reference if so.
@shirishnagarnz : We would recommend you to open new issue for any more questions you have and not mix different issues on same thread. This is just to ensure that anybody looking for a resolution of the initial issue can search it easily and is not diverted with some other discussion on the thread.
@shirishnagarnz : We would recommend you to open new issue for any more questions you have and not mix different issues on same thread. This is just to ensure that anybody looking for a resolution of the initial issue can search it easily and is not diverted with some other discussion on the thread.
Thanks @jayaranigarg . Logged as https://github.com/Microsoft/testfx/issues/576
@shirishnagarnz as Jaya mentioned we dont have this support out of the box today in mstest framework. We are open to contributions on this one.
I also found this link https://automationrhapsody.com/soft-assertions-c-unit-testing-frameworks-mstest-nunit-xunit-net/ which uses FluentAssertions to implement soft assert. you can use that in your test.
My ideal proposal would be to allow an assert
parameter.
[TestMethod] public void MyTest ( Assertions assert)
assert
parameter would be populated by the framework.Assertions
class would expose all of the same methods as the Assert
class. assert
parameter after the function call ends. assert.Checkpoint()
would immediately throw an exception if there are any captured failures. This would allow for existing the test early when there are known problems.If this is too much, option B would be:
[TestMethod] public void MyTest ( Assertions assert)
{
using var assert = new Assertions();
//at the end of the function, assert's Dispose method would throw if there are any captured failures
}
Option B is not as good as option A because option A can capture a the regular exception along with the captured failures and report them together.
Option B will hide the regular exception because assert
has to throw its own exception in the finally clause of the using
blocks.
@shirishnagarnz @Grauenwolf I have also checked and confirm this is not available at the moment. I don't think it would be super complex to implement but we would need to design this correctly.
Out of curiosity, are you expecting to have some tests of a class to be "soft" and the other "hard" or would it be a good compromise for you to say all tests of a given test class would be doing "soft" assertions?
Honestly, I could say. This is something that we should have people try both ways to see which is better.
There is a risk of confusion if soft and hard asserts are mixed.
But if they can't be mixed, it may make some tests harder to write.
That said, we may not have a choice. It may turn out that the syntax dictates that both have to be offered.
I would strongly appreciate this feature. Without this simple feature, the quality of tests is much lower.
What would make the most sense for me is:
Assert.IsNotNull(x); // hard assertion, so that you know that x is not null, otherwise the test logic below would break
Assert.Softly(softAssert => {
softAssert.AreEqual("x",x.A);
softAssert.AreEqual("y",x.B);
});
But then the question is if we can somehow allow people to retain their existing custom assertions, so they can be still used.
I wouldn't worry about confusion of mixing assertions since the way above is explicit and it is also something that a lot of people are used to - done in most of the test frameworks.
I can see this working nicely with async local as well. it would be nice to allow soft assertions by setting that affects more than one test, and have a local opt-out.
Like
var requestResult = call server; // Hard assertion to guard my test from failing in 10 places when I did not even finish the request: Assert.AreEqual(200, requestResult.ReturnCode, throw: true);
soft assertions Assert.HaveCount(3, requestResult.Users); Assert.AreEqual("jakub", requestResult.Users[0].Name); etc.
Description
Steps to reproduce
Expected behavior
Actual behavior
Environment