reqnroll / Reqnroll

Open-source Cucumber-style BDD test automation framework for .NET.
https://reqnroll.net
BSD 3-Clause "New" or "Revised" License
310 stars 32 forks source link

[Error] The active test run was aborted. Reason: Test host process crashed : Stack overflow #71

Closed orifn1 closed 1 month ago

orifn1 commented 4 months ago

Reqnroll Version

1.0.1

Which test runner are you using?

MSTest

Test Runner Version Number

3.0.2

.NET Implementation

.NET 7.0

Test Execution Method

Visual Studio Test Explorer

Content of reqnroll.json configuration file

{ "$schema": "https://schemas.reqnroll.net/reqnroll-config-latest.json",

"trace": { "stepDefinitionSkeletonStyle": "RegexAttribute" } }

Issue Description

[3/6/2024 1:15:42.005 PM] [Error] The active test run was aborted. Reason: Test host process crashed : Stack overflow.
   at Reqnroll.Bindings.AsyncMethodHelper.IsTaskOfT(System.Type, System.Type ByRef)
   at Reqnroll.Bindings.AsyncMethodHelper.IsAwaitableOfT(System.Type, System.Type ByRef)
   at Reqnroll.Bindings.AsyncMethodHelper.GetAwaitableReturnType(Reqnroll.Bindings.Reflection.IBindingMethod)
   at Reqnroll.Bindings.StepArgumentTypeConverter.CanConvert(Reqnroll.Bindings.IStepArgumentTransformationBinding, System.Object, Reqnroll.Bindings.Reflection.IBindingType)
   at Reqnroll.Bindings.StepArgumentTypeConverter+<>c__DisplayClass5_0.<GetMatchingStepTransformation>b__0(Reqnroll.Bindings.IStepArgumentTransformationBinding)
   at System.Linq.Enumerable+WhereListIterator`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ToArray()
   at Reqnroll.Bindings.StepArgumentTypeConverter.GetMatchingStepTransformation(System.Object, Reqnroll.Bindings.Reflection.IBindingType, Boolean)
   at Reqnroll.Bindings.StepArgumentTypeConverter+<ConvertAsync>d__6.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Reqnroll.Bindings.StepArgumentTypeConverter+<ConvertAsync>d__6, Reqnroll, Version=1.0.0.0, Culture=neutral, PublicKeyToken=611ce36403091019]](<ConvertAsync>d__6 ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Start[[Reqnroll.Bindings.StepArgumentTypeConverter+<ConvertAsync>d__6, Reqnroll, Version=1.0.0.0, Culture=neutral, PublicKeyToken=611ce36403091019]](<ConvertAsync>d__6 ByRef)
   at Reqnroll.Bindings.StepArgumentTypeConverter.ConvertAsync(System.Object, Reqnroll.Bindings.Reflection.IBindingType, System.Globalization.CultureInfo)
   at Reqnroll.Bindings.StepArgumentTypeConverter+<DoTransformAsync>d__7.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Reqnroll.Bindings.StepArgumentTypeConverter+<DoTransformAsync>d__7, Reqnroll, Version=1.0.0.0, Culture=neutral, PublicKeyToken=611ce36403091019]](<DoTransformAsync>d__7 ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Start[[Reqnroll.Bindings.StepArgumentTypeConverter+<DoTransformAsync>d__7, Reqnroll, Version=1.0.0.0, Culture=neutral, PublicKeyToken=611ce36403091019]](<DoTransformAsync>d__7 ByRef)
   at Reqnroll.Bindings.StepArgumentTypeConverter.DoTransformAsync(Reqnroll.Bindings.IStepArgumentTransformationBinding, System.Object, System.Globalization.CultureInfo)
   at Reqnroll.Bindings.StepArgumentTypeConverter+<ConvertAsync>d__6.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Reqnroll.Bindings.StepArgumentTypeConverter+<ConvertAsync>d__6, Reqnroll, Version=1.0.0.0, Culture=neutral, PublicKeyToken=611ce36403091019]](<ConvertAsync>d__6 ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Start[[Reqnroll.Bindings.StepArgumentTypeConverter+<ConvertAsync>d__6, Reqnroll, Version=1.0.0.0, Culture=neutral, PublicKeyToken=611ce36403091019]](<ConvertAsync>d__6 ByRef)
   at Reqnroll.Bindings.StepArgumentTypeConverter.ConvertAsync(System.Object, Reqnroll.Bindings.Reflection.IBindingType, System.Globalization.CultureInfo)
   at Reqnroll.Bindings.StepArgumentTypeConverter+<DoTransformAsync>d__7.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Reqnroll.Bindings.StepArgumentTypeConverter+<DoTransformAsync>d__7, Reqnroll, Version=1.0.0.0, Culture=neutral, PublicKeyToken=611ce36403091019]](<DoTransformAsync>d__7 ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Start[[Reqnroll.Bindings.StepArgumentTypeConverter+<DoTransformAsync>d__7, Reqnroll, Version=1.0.0.0, Culture=neutral, PublicKeyToken=611ce36403091019]](<DoTransformAsync>d__7 ByRef)
   at Reqnroll.Bindings.StepArgumentTypeConverter.DoTransformAsync(Reqnroll.Bindings.IStepArgumentTransformationBinding, System.Object, System.Globalization.CultureInfo)
   at Reqnroll.Bindings.StepArgumentTypeConverter+<ConvertAsync>d__6.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Reqnroll.Bindings.StepArgumentTypeConverter+<ConvertAsync>d__6, Reqnroll, Version=1.0.0.0, Culture=neutral, PublicKeyToken=611ce36403091019]](<ConvertAsync>d__6 ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Start[[Reqnroll.Bindings.StepArgumentTypeConverter+<ConvertAsync>d__6, Reqnroll, Version=1.0.0.0, Culture=neutral, PublicKeyToken=611ce36403091019]](<ConvertAsync>d__6 ByRef)
   at Reqnroll.Bindings.StepArgumentTypeConverter.ConvertAsync(System.Object, Reqnroll.Bindings.Reflection.IBindingType, System.Globalization.CultureInfo)
   at Reqnroll.Bindings.StepArgumentTypeConverter+<DoTransformAsync>d__7.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Reqnroll.Bindings.StepArgumentTypeConverter+<DoTransformAsync>d__7, Reqnroll, Version=1.0.0.0, Culture=neutral, PublicKeyToken=611ce36403091019]](<DoTransformAsync>d__7 ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Start[[Reqnroll.Bindings.StepArgumentTypeConverter+<DoTransformAsync>d__7, Reqnroll, Version=1.0.0.0, Culture=neutral, PublicKeyToken=611ce36403091019]](<DoTransformAsync>d__7 ByRef)
   at Reqnroll.Bindings.StepArgumentTypeConverter.DoTransformAsync(Reqnroll.Bindings.IStepArgumentTransformationBinding, System.Object, System.Globalization.CultureInfo)
   at Reqnroll.Bindings.StepArgumentTypeConverter+<ConvertAsync>d__6.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Reqnroll.Bindings.StepArgumentTypeConverter+<ConvertAsync>d__6, Reqnroll, Version=1.0.0.0, Culture=neutral, PublicKeyToken=611ce36403091019]](<ConvertAsync>d__6 ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Start[[Reqnroll.Bindings.StepArgumentTypeConverter+<ConvertAsync>d__6, Reqnroll, Version=1.0.0.0, Culture=neutral, PublicKeyToken=611ce36403091019]](<ConvertAsync>d__6 ByRef)
   at Reqnroll.Bindings.StepArgumentTypeConverter.ConvertAsync(System.Object, Reqnroll.Bindings.Reflection.IBindingType, System.Globalization.CultureInfo)
   at Reqnroll.Bindings.StepArgumentTypeConverter+<DoTransformAsync>d__7.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Reqnroll.Bindings.StepArgumentTypeConverter+<DoTransformAsync>d__7, Reqnroll, Version=1.0.0.0, Culture=neutral, PublicKeyToken=611ce36403091019]](<DoTransformAsync>d__7 ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Start[[Reqnroll.Bindings.StepArgumentTypeConverter+<DoTransformAsync>d__7, Reqnroll, Version=1.0.0.0, Culture=neutral, PublicKeyToken=611ce36403091019]](<DoTransformAsync>d__7 ByRef)
   at Reqnroll.Bindings.StepArgumentTypeConverter.DoTransformAsync(Reqnroll.Bindings.IStepArgumentTransformationBinding, System.Object, System.Globalization.CultureInfo)
   at Reqnroll.Bindings.StepArgumentTypeConverter+<ConvertAsync>d__6.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Reqnroll.Bindings.StepArgumentTypeConverter+<ConvertAsync>d__6, Reqnroll, Version=1.0.0.0, Culture=neutral, PublicKeyToken=611ce36403091019]](<ConvertAsync>d__6 ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Start[[Reqnroll.Bindings.StepA

Steps to Reproduce

just run existing test(migrated from Specflow)

Link to Repro Project

No response

gasparnagy commented 4 months ago

@orifn1 Could you please share what [StepArgumentTransformation]s do you have in that project?

It seems to be similar what @kaylumah reported at https://github.com/kaylumah/hosting/pull/552#issuecomment-1935668615

orifn1 commented 4 months ago

I have a lot - 41

  [StepArgumentTransformation(@"^examples.(\S+)\s*[or]*\s*(\S*)")]
  public string ChangeParameterInBackground(string param, string defaultValue)
  {
      return _scenarioContext.ScenarioInfo.Arguments.Contains(param)
          ? (string)_scenarioContext.ScenarioInfo.Arguments[param]
          : defaultValue;
  }

with 'null' value:

  [StepArgumentTransformation(@"^(\<null\>)$")]
  public decimal? NullStringNullTransformerDecimal(string key)
  {
      return null;
  }
gasparnagy commented 4 months ago

@orifn1 The ones with string return type might be the tricky ones. Can you temporarily comment these ones out and rerun (obviously the tests will fail, but hopefully not with a stack overflow.

I will also check it, but I can do it only tomorrow.

orifn1 commented 4 months ago

It has not helped, I commented out both but the other 39 StepArgumentTransformation were not commented)

kaylumah commented 4 months ago

@orifn1 do you have a link to the repo containing all these transformations? there might be other problematic ones.

For me it was StepTransformation from string to string; but Table transformation to objects is working

orifn1 commented 4 months ago

no, there is no public repo

I commented out several StepTransformations, and some tests passed, but not all. So, I believe it is a bug that should be fixed

orifn1 commented 4 months ago

looks like this code is source of the problem: image Unfortunately, I can't build reqnroll solution locally for now because I have problems with the framework version and do not have enough time to fix it

gasparnagy commented 4 months ago

looks like this code is source of the problem

How did you figure that out? (It might be relevant for the fix as well.)

orifn1 commented 4 months ago

I am not aware, I just investigated stack trace)

orifn1 commented 4 months ago

looks like this code is source of the problem

How did you figure that out? (It might be relevant for the fix as well.)

you were right - the problem in this transformation

 [StepArgumentTransformation]
  public string CustomStringToStringTransformer(string str)
  {
      if (NullValueRegex.IsMatch(str))
      {
          return null;
      }

      return TryReplaceBodyPattern(str, _domainContext);
  }

this method returned it recursively because check CanConvert

image

I will try to handle it with your suggestion in the mentioned issue https://github.com/SpecFlowOSS/SpecFlow/issues/2561

gasparnagy commented 4 months ago

@orifn1 thank you for looking into that. I didn't even remember that I made that comment. :) If you come up with any good workaround, please post it here.

kaylumah commented 4 months ago

Interessting, that is very similar conversion (so probably same issue)

[StepArgumentTransformation]
public static string ToNullableString(string value)
{
   return Constants.NullIndicator.Equals(value, System.StringComparison.Ordinal) ? null : value;
}

Never knew that string -> string was not supported; Is that something changed between V3 / V4 (on which Reenrol is based) or did it work by accident on V3?

gasparnagy commented 4 months ago

No this is a regression in SpecFlow v3->v4 or in the Reqnroll porting (unfortunately I had to touch the converters, so it can happen that it was reintroduced during the forking). :(

gasparnagy commented 4 months ago

If any of you could make a simple check with SpecFlow v4-beta to see if the regression was already present there, it would help the investigation.

orifn1 commented 3 months ago

I have created a very small workaround to fix the stack overflow issue but can not push it into the repo because I do not have access I just skip transformation if it was applied previously It is not the best solution but it works) image

gasparnagy commented 3 months ago

@orifn1 thx. I will try to integrate this to the codebase next week.

adrichem commented 3 months ago

I'm also running into this issue, this transform reproduces it. For us its a blocker to migrate away from SpecFlow.

//Replaces placeholders in the input with correct run time values.  
[StepArgumentTransformation]
public string Transform(string input) => Parser.Parse(input);