search

paulegradie / Sailfish

Sailfish - a production friendly performance benchmark runner for .NET
https://paulgradie.com/Sailfish/
MIT License
11 stars 4 forks source link

Exception from a test SailfishMethod does not contain the TestCaseId #171

Closed shaun-od closed 1 month ago

shaun-od commented 1 month ago

Describe the bug When an exception is thrown in an actual test (SailfishMethod) the exception appears in the CompiledTestCaseResult but without a TestCaseId. This is similar to #167

To Reproduce Steps to reproduce the behavior:

  1. Run the existing Sailfish Test OnlySailfishMethodThrows
  2. Check the result object returned result.ExecutionSummaries[0].CompiledTestCaseResults[0].TestCaseId is null

Expected behavior This use to be returned in V1.4.7. I have just upgraded to 2.0.97 and it is now null.

I do wonder if some other states are also missing the TestCaseId that shouldn't be but I haven't been able to look.

paulegradie commented 1 month ago

Thanks again @shaun-od 👍 I'll get this addressed and pushed out tomorrow.

paulegradie commented 1 month ago

An easy way to check for other scenarios with the missing testId will be to assert on that property in the existing lifecycle method tests. In additional, I'll just double check instantiations of the TestCaseExecutionResult object and see how data is passed to its contructor.

I'll add that, and any missing lifecycle tests.

paulegradie commented 1 month ago

Addressed in https://github.com/paulegradie/Sailfish/pull/173