search

timschlechter / SpecResults.WebApp

Writes an interactive, responsive, client-side web application, in which users can browse and search features, scenarios and steps.
https://www.nuget.org/packages/SpecFlow.Reporting.WebApp/
MIT License
9 stars 6 forks source link

Missing Log entries & Empty results are schown as "OK" #3

Open michael9876 opened 9 years ago

michael9876 commented 9 years ago

There seems to be a problem with uncaught exceptions. NUnit collects them all and reports them in their result.xml, but they are not found in the WebApp.

Here is an example for such an exception taken from the NUnit result xml:

                                  <stack-trace><![CDATA[

Server stack trace: at System.Threading.Thread.InternalCrossContextCallback(Context ctx, IntPtr ctxID, Int32 appDomainID, InternalCrossContextDelegate ftnToCall, Object[] args) at System.Runtime.Remoting.Channels.CrossContextChannel.SyncProcessMessage(IMessage reqMsg)

Exception rethrown at [0]: at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg) at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type) at .When at lambda_method(Closure , IContextManager , String ) at TechTalk.SpecFlow.Bindings.BindingInvoker.InvokeBinding(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, TimeSpan& duration) at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStepMatch(BindingMatch match, Object[] arguments) at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStep(StepInstance stepInstance) at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.Step(StepDefinitionKeyword stepDefinitionKeyword, String keyword, String text, String multilineTextArg, Table tableArg) at TechTalk.SpecFlow.TestRunner.When(String text, String multilineTextArg, Table tableArg, String keyword) at () in .feature:line <> ]]>

As a sidenode: The GlueCode and the selenium related code are in an external library.

In the WebApp the scenario result is 'OK' and when you open it, there are no Results in there.

Incase it is not possible to actually catch this exception (there are other examples as well) and write it to the result, the WebApp should definately default to: "When a scenario doesnt contain any steps, the scenario result should definately NOT be 'OK' but 'Error'"

The following result data (generated as a result of the unlogged exception above) leads to such a dangerous behavior: { "given": { "result": "OK", "steps": [], "start_time": "0001-01-01T00:00:00", "end_time": "0001-01-01T00:00:00" }, "when": { "result": "OK", "steps": [], "start_time": "2014-12-08T18:55:21.0951765+01:00", "end_time": "2014-12-08T18:55:22.7046705+01:00" }, "then": { "result": "OK", "steps": [], "start_time": "2014-12-08T18:55:22.7046705+01:00", "end_time": "2014-12-08T18:55:23.0354289+01:00" }, "result": "OK", "tags": [], "title": "my scenario title", "start_time": "2014-12-08T18:55:21.0951765+01:00", "end_time": "2014-12-08T18:55:23.0354289+01:00" },

Maybe even a step without a title should be automatically marked as 'Error'

thanks alot for your support - feel free to contact me back

jacdevos commented 9 years ago

I get the same issue - bit of a showstopper