Getting unexpected Unhandled NullReferenceException

[savehansson]

When starting Alive on a specific unit test method I get this exception:

There are no clues to why (or where in the code it is thrown), not even in the output window:

[Error v1.4-vs14.0] Unhandled NullReferenceException: Object reference not set to an instance of an object.
[Error v1.4-vs14.0] Stack trace: 
   at Edison.Helpers.Tests.StringHelperTest.TillRenTextTest()
   at Edison.Helpers.Tests.StringHelperTest.__aliveIntermediateType.Edison_Helpers_Tests_StringHelperTest__TillRenTextTest__aliveIntermediate()

As you can see from the screen dump the test runs fine in VS, without any exception.

Moving to the method tested and choosing Continue doesn't help much. Same info there and the arguments and return values for all the three test cases looks normal. None of them are null by the way.

What am I missing?

[AmadeusW]

I'm very sorry about it.

Do you get this exception with all three test cases, or just this one? The stack trace indeed doesn't say much - your method was executed and something went wrong there. So far I tested if Environment.NewLine might be the culprit, but it works fine. Is it ok for you to share your test method? You can post it here or email me at amadeus@comealive.io

[savehansson]

After having taken a closer look at my code, I think I've isolated the issue. The exception is thrown when I use the TestContext Class to write trace messages while the test is running. These messages can be viewed as Output in the Test Explorer:

I noticed that my one (1) test method actually contains 5 individual tests (Assert), but Alive only counts the three first. The others are "grayed out" when running Alive indicating that it is the fourth test that introduces the problem. That one (and the fifth as well) makes a call to the method under test, writes some output using TestContext.WriteLine(...) and then calls Assert.AreEqual(expected, actual).

The problem can be reproduced like so:

[TestClass]
public partial class AliveTest
{
    public TestContext TestContext { get; set; }

    [TestMethod()]
    public void AliveTest()
    {
        TestContext.WriteLine("");
    }
}

Hope that helps!

[AmadeusW]

Yup, this is it. TestContext is abstract and its known implementations are internal, so we couldn't use them. We could supply our own TestContext but initially we decided against doing it. Our rationale was that supplying TestContext would turn a null ref exception into a runtime exception:

Suppose user's code has TestContext.DeploymentDirectory. If we provided TestContext, there will be a runtime exception when user attempts to use the invalid data from DeploymentDirectory.

However, now I see that you're using the trace capability. I will try to create TestContext that provides as much information as possible and accepts trace messages. Hopefully this will fix your issue.

[AmadeusW]

@hawkunsh do you use the [AssemblyInitialize] attribute in your tests?

[savehansson]

Not using [AssemblyInitialize].

[AmadeusW]

@hawkunsh I've got code that builds an instance of TestContext and uses it while running your tests. There are no more null reference exceptions.

However, TestContext has a property RequestedPage whose type is defined in System.Web assembly. We don't want to add references to user's project because we wouldn't be able to reliably support different code bases across different versions of Visual Studio. This property causes Alive to not work in certain situations, like the commented out assignment:

We're working on a more performant way to retrieve object's properties that won't be affected by these troublesome properties. It's approximately a week away from being done.

If you're just using TestContext.WriteLine, my current solution works and should fix your issue. Please send me an email at amadeus@codeconnect.io and I'll send you Alive 1.4.2 with a fix for your issue.

[AmadeusW]

I sent it. Let me know if you haven't received it (.vsix is a .zip with .dll files, which may cause the mail providers to reject the email)

[savehansson]

You also published 1.4.2 to the Extensions Galley so I updated it from there. Ok?

Haven't tested it yet though.

[AmadeusW]

No, that's won't work. The version at the extensions gallery doesn't have the TestContext fix. I'm emailing you a version I built just for you. We're not releasing this fix to the public yet because we know that it breaks in certain corner cases (but it should work fine for you)

[AmadeusW]

The email bounced. You can get the extension from here: https://codeconnectcdn.blob.core.windows.net/cdn/PrivateAliveReleases/Alive.1.4.2.withTestContextFix.vsix

This version will inject an instance TestContext to the TestContext property and into the ClassInitialize method.

I did some basic tests with the TestContext.WriteLine and it worked fine. Let me know if it doesn't work in your code!

P.S. You will need to uninstall current Alive to install this version (because the version number is the same)

[savehansson]

It works fine now. Thanks! Will this be included in 1.4.3?