(MSTest Support) Unquote eats exceptions causing failed tests to pass

[rookboom]

I have the following MSTest using F#:

    [<TestMethod>]
    member m.``Div by zero``() =
        let divBy x y = x / y
        test <@ 10 = divBy 10 0 @>

If I run all my tests they all pass including this one. If I run only this one test from the Test Explorer it does fail though. Even when running tests from the commandline (using FAKE) the test still passes.

If I move the function outside of the quotation like this:

        [<TestMethod>]
        member m.``Div by zero``() =
            let divBy x y = x / y
            let answer = divBy 10 0
            test <@ 10 = answer @>

Then it does fail consistently so it seems that if I have any expression within a quotation that throws an exception, I will not know about it.

This is really problematic because I don't know whether I can trust my unit tests.

I am using VS 2017 and Unquote 3.2.0

[stephen-swensen]

Hi @rookboom - unfortunately I am not able to reproduce the issue you are experiencing, do you spot anything that might be different / special about your project setup vs. mine (if you could provide a repo project that would be great)?

• VS 2017 (Version 15.2)
• Unquote 3.2.0
• MSTest.TestAdapter 1.1.18
• MSTest.TestFramework 1.1.18
• .NET Framework 4.5.2
• F# 4.1 (FSharp.Core 4.4.1.0)

[rookboom]

UnquoteProblem.zip

Hi Stephen,

I attached a minimal example that reproduces the error. It looks like there is some weird interaction when instantiating an instance of FsiEvaluationSession. (I am testing some in-process script evaluation). Unzip the attached file and run restore.cmd to restore all the nuget packages. Then open the solution and rebuild all. Then run all unit tests. (There is only one).

On my machine the test passes but should fail.

If I replace

        test <@ "snarfunkle" = e @>

with

        Assert.Fail()

then the test does fail as it should.

This is a strange one.

Thanks

Johan

[rookboom]

Turns out switching from MSTest to NUnit fixes this issue.

[stephen-swensen]

Closing this issue for it's age and because it wasn't reproducible at the time.