Closed clrudolphi closed 3 months ago
@clrudolphi looks good.
Notes for the system tests:
GenerationTestBase
class. That way it is even ensured that it is checked with all 3 test execution framework. The project name setup line (_projectsDriver.CreateProject(...)
) you can simply into the test (before the call to AddScenario
). AddScenario
call within the same test.)It seems from the CI that a few of the unit tests in Reqnroll.GeneratorTests needs to be adjusted. Please check that and click on the "Ready for review" if you think it is ready. I will do a final review after that.
@clrudolphi thank you!
Do you have any feedback on how system tests worked? Any challenges you have faced?
The only minor hiccup was that AddScenario() creates a (default) feature, so that you cannot call AddScenario more than one time per test.
It is a little challenging to get control over feature structure and feature and file naming but not insurmountable. Use of the 'load feature from resource' might be the better option for those testing situations.
Once I had the prerequisite framework versions installed, all the tests ran smoothly and pretty quickly.
@gasparnagy Should we also cover situations in which the user has placed the name of the testing framework (such as XUnit) in their namespace? See for example this bug report on SpecFlow here
I've found an alternative approach that works for when either/both of Reqnroll and or the testing framework name is mentioned in the user's Namespace. Let me know what you think.
using Reqnroll.xUnit.ReqnrollPlugin;
using Xunit.Abstractions;
The resulting generated code file would look something like this:
// ------------------------------------------------------------------------------
// <auto-generated>
// This code was generated by Reqnroll (https://www.reqnroll.net/).
// Reqnroll Version:1.0.0.0
// Reqnroll Generator Version:1.0.0.0
//
// Changes to this file may cause incorrect behavior and will be lost if
// the code is regenerated.
// </auto-generated>
// ------------------------------------------------------------------------------
#region Designer generated code
using Reqnroll.xUnit.ReqnrollPlugin;
using Xunit.Abstractions;
#pragma warning disable
namespace RnRGH81XUnitNet6.Features.Reqnroll.Xunit
{
using System;
using System.Linq;
[System.CodeDom.Compiler.GeneratedCodeAttribute("Reqnroll", "1.0.0.0")]
[System.Runtime.CompilerServices.CompilerGeneratedAttribute()]
public partial class ColorEnumTestingFeature : object, IClassFixture<ColorEnumTestingFeature.FixtureData>, IAsyncLifetime
{
private static ITestRunner testRunner;
private static string[] featureTags = ((string[])(null));
private ITestOutputHelper _testOutputHelper;
#line 1 "ColorEnumTest.feature"
#line hidden
public ColorEnumTestingFeature(ColorEnumTestingFeature.FixtureData fixtureData, ITestOutputHelper testOutputHelper)
{
this._testOutputHelper = testOutputHelper;
}
public static async System.Threading.Tasks.Task FeatureSetupAsync()
{
testRunner = TestRunnerManager.GetTestRunnerForAssembly(null, XUnitParallelWorkerTracker.Instance.GetWorkerId());
FeatureInfo featureInfo = new FeatureInfo(new System.Globalization.CultureInfo("en-US"), "Features", "Color Enum Testing", null, ProgrammingLanguage.CSharp, featureTags);
await testRunner.OnFeatureStartAsync(featureInfo);
}
public static async System.Threading.Tasks.Task FeatureTearDownAsync()
{
string testWorkerId = testRunner.TestWorkerId;
await testRunner.OnFeatureEndAsync();
testRunner = null;
XUnitParallelWorkerTracker.Instance.ReleaseWorker(testWorkerId);
}
public async System.Threading.Tasks.Task TestInitializeAsync()
{
}
public async System.Threading.Tasks.Task TestTearDownAsync()
{
await testRunner.OnScenarioEndAsync();
}
public void ScenarioInitialize(ScenarioInfo scenarioInfo)
{
testRunner.OnScenarioInitialize(scenarioInfo);
testRunner.ScenarioContext.ScenarioContainer.RegisterInstanceAs<ITestOutputHelper>(_testOutputHelper);
}
public async System.Threading.Tasks.Task ScenarioStartAsync()
{
await testRunner.OnScenarioStartAsync();
}
public async System.Threading.Tasks.Task ScenarioCleanupAsync()
{
await testRunner.CollectScenarioErrorsAsync();
}
async System.Threading.Tasks.Task IAsyncLifetime.InitializeAsync()
{
await this.TestInitializeAsync();
}
async System.Threading.Tasks.Task IAsyncLifetime.DisposeAsync()
{
await this.TestTearDownAsync();
}
[SkippableFactAttribute(DisplayName="Name a color enum")]
[TraitAttribute("FeatureTitle", "Color Enum Testing")]
[TraitAttribute("Description", "Name a color enum")]
public async System.Threading.Tasks.Task NameAColorEnum()
{
string[] tagsOfScenario = ((string[])(null));
System.Collections.Specialized.OrderedDictionary argumentsOfScenario = new System.Collections.Specialized.OrderedDictionary();
ScenarioInfo scenarioInfo = new ScenarioInfo("Name a color enum", null, tagsOfScenario, argumentsOfScenario, featureTags);
#line 3
this.ScenarioInitialize(scenarioInfo);
#line hidden
if ((TagHelper.ContainsIgnoreTag(tagsOfScenario) || TagHelper.ContainsIgnoreTag(featureTags)))
{
testRunner.SkipScenario();
}
else
{
await this.ScenarioStartAsync();
#line 4
await testRunner.GivenAsync("a color of \'Red\'", ((string)(null)), ((Table)(null)), "Given ");
#line hidden
#line 5
await testRunner.WhenAsync("I click the abc element and wait", ((string)(null)), ((Table)(null)), "When ");
#line hidden
#line 6
await testRunner.ThenAsync("the color of \'Black\' should be OK", ((string)(null)), ((Table)(null)), "Then ");
#line hidden
}
await this.ScenarioCleanupAsync();
}
[SkippableFactAttribute(DisplayName="2nd scenario")]
[TraitAttribute("FeatureTitle", "Color Enum Testing")]
[TraitAttribute("Description", "2nd scenario")]
public async System.Threading.Tasks.Task _2NdScenario()
{
string[] tagsOfScenario = ((string[])(null));
System.Collections.Specialized.OrderedDictionary argumentsOfScenario = new System.Collections.Specialized.OrderedDictionary();
ScenarioInfo scenarioInfo = new ScenarioInfo("2nd scenario", null, tagsOfScenario, argumentsOfScenario, featureTags);
#line 8
this.ScenarioInitialize(scenarioInfo);
#line hidden
if ((TagHelper.ContainsIgnoreTag(tagsOfScenario) || TagHelper.ContainsIgnoreTag(featureTags)))
{
testRunner.SkipScenario();
}
else
{
await this.ScenarioStartAsync();
#line 9
await testRunner.GivenAsync("a color of \'Black\'", ((string)(null)), ((Table)(null)), "Given ");
#line hidden
#line 10
await testRunner.ThenAsync("the color of \'Black\' should be OK", ((string)(null)), ((Table)(null)), "Then ");
#line hidden
}
await this.ScenarioCleanupAsync();
}
[System.CodeDom.Compiler.GeneratedCodeAttribute("Reqnroll", "1.0.0.0")]
[System.Runtime.CompilerServices.CompilerGeneratedAttribute()]
public class FixtureData : object, IAsyncLifetime
{
async System.Threading.Tasks.Task IAsyncLifetime.InitializeAsync()
{
await ColorEnumTestingFeature.FeatureSetupAsync();
}
async System.Threading.Tasks.Task IAsyncLifetime.DisposeAsync()
{
await ColorEnumTestingFeature.FeatureTearDownAsync();
}
}
}
}
#pragma warning restore
#endregion
This fix changes all references to Reqnroll types in the generated code as global:: references.
Created a utility method in CodeDomHelper to take a Type and return a string as "global::" + type full name.
Replaced all occurrences of typeof() with an invocation of the utility method. This switches the use of the CodeDom Type Reference constructors from using a Type to using a String (should have no noticeable side-effects).
Replaced a few occurrences where the generators were already using the string name of the type.
Tests are provided for a simple scenario and one containing a DataTable (as suggested). Don't yet have tests for DocStrings, Scenario Outlines, Rules or Backgrounds. Welcome suggestions on which of these to do and how to proceed (using a resource file?).
Not sure what, if anything, needs to be done to support VB.
Types of changes
Checklist: