machine / machine.specifications

Machine.Specifications is a Context/Specification framework for .NET that removes language noise and simplifies tests.
MIT License
885 stars 178 forks source link

Exceptions thrown from Async test delegates do not have stack traces #521

Closed neilrees closed 4 months ago

neilrees commented 5 months ago

If an exception is thrown from a async function the stack trace is not captured by the test runner.

For example:

Establish context = async () =>
{
    await Task.Delay(1);
    throw new Exception("Test exception");
};

Produces as test output:

System.Exception: Test exception

System.Exception
Test exception
  Exception doesn't have a stacktrace

Where as a syncronous method:

Establish context = () =>
{
    throw new Exception("Test exception");
};

Produces:

System.Exception: Test exception

System.Exception
Test exception
   at MspecExceptions.ExceptionTests.it_tests.<>c.<.ctor>b__2_1() in W:\skunkworks\MspecExceptions\MspecExceptions\ExceptionTests.cs:line 65
   at InvokeStub_It.Invoke(Object, Object, IntPtr*)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)

Behavior is the same in Rider, dotnet test and Visual Studio.

Tested with:

<ItemGroup>
    <PackageReference Include="Machine.Specifications" Version="1.1.1" />
    <PackageReference Include="Machine.Specifications.Runner.VisualStudio" Version="2.10.2" />
    <PackageReference Include="Machine.Specifications.Should" Version="1.0.0" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.10.0" />
</ItemGroup>

Here's a test class that repros the issue for Establish, Because and It delegates:

public class ExceptionTests
{
    public class when_establish_throws
    {
        public class when_establish_is_async
        {
            Establish context = async () =>
            {
                await Task.Delay(1);
                throw new Exception("Test exception");
            };

            It never_gets_here = () => { };
        }

        public class when_establish_is_sync
        {
            Establish context = () =>
            {
                throw new Exception("Test exception");
            };

            It never_gets_here = () => { };
        }
    }
    public class when_because_throws
    {

        public class when_because_is_async
        {
            Because of_an_unhandled_exception = async () =>
            {
                await Task.Delay(1);
                throw new Exception("Test exception");
            };

            It never_gets_here = () => { };
        }

        public class when_because_is_sync
        {
            Because of_an_unhandled_exception = () =>
            {
                throw new Exception("Test exception");
            };

            It never_gets_here = () => { };
        }
    }

    public class when_it_throws
    {
        It is_async = async () =>
        {
            await Task.Delay(1);
            throw new Exception("Test exception");
        };

        It is_sync = () =>
        {
            throw new Exception("Test exception");
        };
    }
}
neilrees commented 5 months ago

Suspect it's due to the exception being re-thrown in DelegateRunner and overwriting the stack trace https://github.com/machine/machine.specifications/blob/f2ded639870ae1aa3fa30480085fbab425b21545/src/Machine.Specifications.Core/Runner/Impl/DelegateRunner.cs#L43

Which subsequently gets entirely filtered out in ExceptionResult: https://github.com/machine/machine.specifications/blob/f2ded639870ae1aa3fa30480085fbab425b21545/src/Machine.Specifications.Core/ExceptionResult.cs#L91

robertcoltheart commented 4 months ago

This has been released as v1.1.2