paulegradie / Sailfish

Sailfish - a production friendly performance benchmark runner for .NET
https://paulgradie.com/Sailfish/
MIT License
11 stars 4 forks source link

Throw more specific exception when test class fails to be instantiated #136

Open paulegradie opened 5 months ago

paulegradie commented 5 months ago

When building tests and a registration is missing or some other issue prevents a test class from being instantiated for a given test case, a vanilla exception is thrown.

It would be ideal to have a more specific (custom?) exception that describes this scenario - which may even be used for additional filtering when reporting test results for example:

        catch (Exception ex)
        {
            exceptionCallback?.Invoke(testCaseEnumerator.Current);

            await DisposeOfTestInstance(testCaseEnumerator.Current);
            testCaseEnumerator.Dispose();
            var msg = $"Error resolving test from {testProvider.Test.FullName}";
            Log.Logger.Fatal(ex, "{Message}", msg);
            if (exceptionCallback is null) throw;

            var testCaseEnumerationException = new TestCaseEnumerationException(ex, $"Failed to create test cases for {testProvider.Test.FullName}");
            return new List<TestCaseExecutionResult>() { new(testCaseEnumerationException) };
        }

Or something to that effect.