Scenario Outline String Problem

[wcdeich4]

I've been having bugs with SpecFlow. In particular when I try to use strings in scenario outlines, it makes at least 1 of the strings part of the method name, which makes it impossible pass multiple string values into that variable for each row of the scenario outline.

[samholder]

It's not clear what the problem actually is. Could you give an example? On 17 Aug 2015 22:54, "wcdeich4" notifications@github.com wrote:

I've been having bugs with SpecFlow. In particular when I try to use strings in scenario outlines, it makes at least 1 of the strings part of the method name, which makes it impossible pass multiple string values into that variable for each row of the scenario outline.

— Reply to this email directly or view it on GitHub https://github.com/techtalk/SpecFlow/issues/473.

[wcdeich4]

Howdy. Well, creating an example for you, I realized the problem is not quite as bad as I thought, but there are some problems, so let me explain. Create the following Scenario Outline:

Feature: SpecFlowFeature1 string processing

@mytag Scenario Outline: String Parsing Given I have entered And Then I entered And Then I also entered When I press combine the strings Then the result should be

Examples: 

| ValueA | ValueB | ValueC | ValueD | | "Hello" | "Big" | "World" | "HelloBigWorld" | | "Goodbye" | "Small" | "World" | "GoodbyeSmallWorld" | | "http://" | "plaza.ufl.edu/" | "gospelsf" | "http://plaza.ufl.edu/gospelsf" |

Now Right Click & "Generate Step Definitions" & select "method name - pascal case" & add a few simple C# lines & you could have something like this:

using System; using TechTalk.SpecFlow;

namespace SpecFlowTests { [Binding] public class SpecFlowFeature1Steps { string combined = "";

    [Given]
    public void GivenIHaveEntered_P0(string p0)
    {
        combined = combined + p0;
    }

    [Given]
    public void GivenThenIEntered_P0(string p0)
    {
        combined = combined + p0;
    }

    [Given]
    public void GivenThenIAlsoEntered_P0(string p0)
    {
        combined = combined + p0;
    }

    [When]
    public void WhenIPressCombineTheStrings()
    {

    }

    [Then]
    public void ThenTheResultShouldBe_P0(string p0)
    {
        NUnit.Framework.Assert.AreEqual(combined, p0);
    }
}

}

However, the Assert fails because the combined string comes out as "Goodbye\"Small\"World\"" where as the expected value p0 comes out as "GoodbyeSmallWorld\""

Although it does work if you Generate the Definition Steps using "Regular expressions in attributes" then it does work; however, I was confused up until just now. This is because, like most C# programmers, I used the NUnitAdapter with NUnit so I can run the tests from Visual Studio's Test Explorer window. And the test shows up as:

StringParsing("\"Goodbye\"","\"Small\"","\"World\"","\"GoodbyeSmallWorld\"",null)

Also, the annotations in the step file come out looking like:

[Given(@"I have entered ""(.*)""")]

Now all of those \"","\" and ""(.*)""" look like invalid C# so I kept thinking I had to remove the extra quotes, and that always made the tests fail, or not even compile. However just now, I tried running it with all those extra quotes that look like invalid C# code & it works for regular expressions in the

One last thing: I would suggest making it possible so the programmer can manually create their own attribute tags to hook the line of Gherkin to the test function, like in Java. When SpecFlow malfunctions, it is almost always on the Step Definition Generation.

[darrencauthon]

If I were to vote for anything, it would be to make the regular expressions the default via 2.0 and watch this stuff vanish.

I've been binding strings to method via regex for many years, with SpecFlow and without, and I've never had these types of problems. And 99.9% of my regex statements have only been "(.*)" wildcards. The majority of the time I used it, I didn't even understand regular expressions!

Gherkin is written with strings.

Regular expressions are meant for string matching.

C# method names are meant for writing computer programs -- not Gherkin.

I think method names like GivenThenIAlsoEntered_P0 are a "BDD impedance mismatch." We're matching strings to C# with arbitrary conventions, and people trip over them, and we hear phrases like "SpecFlow malfunctions." (and for good reason).

But SpecFlow is great, it's never "malfunctioned" like this for me... so why not keep most people on the happy regex path? Let all the people who think they read code better than English (spoiler alert: they don't) flip the flag in their config and deal with the consequences.

[samholder]

I think it would work if you excluded the " from the examples. Specflow will pass them as strings anyway, and from your example it seems that the only issue is that you have is the extra quotes.

As Darren said I rarely have these issues and don't know if this is an issue that really exists.

Sam On 18 Aug 2015 19:51, "wcdeich4" notifications@github.com wrote:

Howdy. Well, creating an example for you, I realized the problem is not quite as bad as I thought, but there are some problems, so let me explain. Create the following Scenario Outline:

Feature: SpecFlowFeature1 string processing

@mytag Scenario Outline: String Parsing Given I have entered And Then I entered And Then I also entered When I press combine the strings Then the result should be

Examples:

| ValueA | ValueB | ValueC | ValueD | | "Hello" | "Big" | "World" | "HelloBigWorld" | | "Goodbye" | "Small" | "World" | "GoodbyeSmallWorld" | | "http://" | "plaza.ufl.edu/" | "gospelsf" | " http://plaza.ufl.edu/gospelsf" |

Now Right Click & "Generate Step Definitions" & select "method name - pascal case" & add a few simple C# lines & you could have something like this:

using System; using TechTalk.SpecFlow;

namespace SpecFlowTests { [Binding] public class SpecFlowFeature1Steps { string combined = "";

[Given]
public void GivenIHaveEntered_P0(string p0)
{
    combined = combined + p0;
}

[Given]
public void GivenThenIEntered_P0(string p0)
{
    combined = combined + p0;
}

[Given]
public void GivenThenIAlsoEntered_P0(string p0)
{
    combined = combined + p0;
}

[When]
public void WhenIPressCombineTheStrings()
{

}

[Then]
public void ThenTheResultShouldBe_P0(string p0)
{
    NUnit.Framework.Assert.AreEqual(combined, p0);
}

}

}

However, the Assert fails because the combined string comes out as "Goodbye\"Small\"World\"" where as the expected value p0 comes out as "GoodbyeSmallWorld\""

Although it does work if you Generate the Definition Steps using "Regular expressions in attributes" then it does work; however, I was confused up until just now. This is because, like most C# programmers, I used the NUnitAdapter with NUnit so I can run the tests from Visual Studio's Test Explorer window. And the test shows up as:

StringParsing("\"Goodbye\"","\"Small\"","\"World\"","\"GoodbyeSmallWorld\"",null)

Also, the annotations in the step file come out looking like:

[Given(@"I have entered ""(.*)""")]

Now all of those \"","\" and ""(.*)""" look like invalid C# so I kept thinking I had to remove the extra quotes, and that always made the tests fail, or not even compile. However just now, I tried running it with all those extra quotes that look like invalid C# code & it works for regular expressions in the

One last thing: I would suggest making it possible so the programmer can manually create their own attribute tags to hook the line of Gherkin to the test function, like in Java. When SpecFlow malfunctions, it is almost always on the Step Definition Generation.

— Reply to this email directly or view it on GitHub https://github.com/techtalk/SpecFlow/issues/473#issuecomment-132294591.

[wcdeich4]

I'll try to keep you up to date.

[darrencauthon]

Even if the quotes are removed, it will only be a matter of time until someone else hits the same issue. This isn't the first time I've seen this. There was a ticket or two regarding this matter in the past, but I don't know if the issue was ever resolved.

And for every ticket, there are probably dozens of people who hit the problem and just eat it.

[wcdeich4]

Just to be clear - it worked with the extra quotes - it did not work when I removed them

[wcdeich4]

Also if you make the examples: | ValueA | ValueB | ValueC | ValueD | | Hello | Big | World | HelloBigWorld | | Goodbye | Small | World | GoodbyeSmallWorld | (without the string quotes),

then it puts the first row of strings into the function names with no parameters at all, so it's impossible to pass multiple rows of data into the test functions

[samholder]

I'm assuming that your given step actually looks like this:

Given I have entered

Which is how I would usually have the steps (but I always use regex, so have never seen this issue) On 18 Aug 2015 20:52, "wcdeich4" notifications@github.com wrote:

Also if you make the examples: | ValueA | ValueB | ValueC | ValueD | | Hello | Big | World | HelloBigWorld | | Goodbye | Small | World | GoodbyeSmallWorld | (without the string quotes),

then it puts the first row of strings into the function names with no parameters at all, so it's impossible to pass multiple rows of data into the test functions

— Reply to this email directly or view it on GitHub https://github.com/techtalk/SpecFlow/issues/473#issuecomment-132317095.

[wcdeich4]

Even with the regex, it did not generate a string parameter, it just made the name "GivenIhaveEnteredHello()". If there is no parameter in the function, you've got a problem. I tried modifying the functions to add parameters, but it crashed.

[wcdeich4]

I was talking with my friends about why I never just tried running the tests with all those autogenerated multiple quotation marks in a row. I think I was kind of expecting errors because I ran into so many problems setting up SpecFlow to begin with. For example, the instructions online said you could start with a console application and then add NUnit & SpecFlow through NuGet, however that did not work for me using Visual Studio Premium 2013. It failed to create a working SpecFlow project several times, and eventually I had to create a UnitTest, convert it from Microsoft Unit Testing framework to NUnit, add the NUnitAdapter & then add SpecFlow for it to work properly.

[samholder]

It won't generate a function with a parameter if you don't include something which indicates that a parameter is required. A step defined as

Given I enter a value

Will not have a parameter as there is nothing in the step which indicates what the parameter is.

The step

Given I have a value 'something'

Will generate a function with a parameter as specflow can tell from the step the the value in quotes is likely to be a parameter (because it's in quotes)

When you use scenario outlines you can match the parameter to the column in the examples using the column name in angle brackets, so this

Given I have a value

Will generate a function with a single string parameter and will pass the value from the column called Value1 from each example to that function.

I use the generation all the time and rely have to change it. On 18 Aug 2015 22:23, "wcdeich4" notifications@github.com wrote:

Even with the regex, it did not generate a string parameter, it just made the name "GivenIhaveEnteredHello()". If there is no parameter in the function, you've got a problem. I tried modifying the functions to add parameters, but it crashed.

— Reply to this email directly or view it on GitHub https://github.com/techtalk/SpecFlow/issues/473#issuecomment-132340080.

[wcdeich4]

Feature: SpecFlowFeature1 string processing

@mytag Scenario Outline: String Parsing Given I have entered And Then I entered And Then I also entered When I press combine the strings Then the result should be

Examples: 

| ValueA | ValueB | ValueC | ValueD | | Hello | Big | World | HelloBigWorld | | Goodbye | Small | World | GoodbyeSmallWorld | | http:// | plaza.ufl.edu/ | gospelsf | http://plaza.ufl.edu/gospelsf |

-----------------------------Generates------------------------------------------------------------

using System; using TechTalk.SpecFlow;

namespace SpecFlowTests { [Binding] public class SpecFlowFeature1Steps { [Given(@"I have entered Hello")] public void GivenIHaveEnteredHello() { ScenarioContext.Current.Pending(); }

    [Given(@"Then I entered Big")]
    public void GivenThenIEnteredBig()
    {
        ScenarioContext.Current.Pending();
    }

    [Given(@"Then I also entered World")]
    public void GivenThenIAlsoEnteredWorld()
    {
        ScenarioContext.Current.Pending();
    }

    [When(@"I press combine the strings")]
    public void WhenIPressCombineTheStrings()
    {
        ScenarioContext.Current.Pending();
    }

    [Then(@"the result should be HelloBigWorld")]
    public void ThenTheResultShouldBeHelloBigWorld()
    {
        ScenarioContext.Current.Pending();
    }
}

}

[samholder]

Given this feature:

Feature: SpecFlowFeature1string processing@mytagScenario Outline: String ParsingGiven I have entered ''And Then I entered ''And Then I also entered ''When I press combine the stringsThen the result should be '' Examples: | ValueA | ValueB | ValueC | ValueD | | Hello | Big | World | HelloBigWorld | | Goodbye | Small | World | GoodbyeSmallWorld | | http:// | plaza.ufl.edu/ | gospelsf | http://plaza.ufl.edu/gospelsf |

I get this generated:

using System;using TechTalk.SpecFlow; namespace ClassLibrary1 { [Binding] public class SpecFlowFeature1Steps { [Given(@"I have entered '(.*)'")] public void GivenIHaveEntered(string p0) { ScenarioContext.Current.Pending(); }

    [Given(@"Then I entered '(.*)'")]
    public void GivenThenIEntered(string p0)
    {
        ScenarioContext.Current.Pending();
    }

    [Given(@"Then I also entered '(.*)'")]
    public void GivenThenIAlsoEntered(string p0)
    {
        ScenarioContext.Current.Pending();
    }

    [When(@"I press combine the strings")]
    public void WhenIPressCombineTheStrings()
    {
        ScenarioContext.Current.Pending();
    }

    [Then(@"the result should be '(.*)'")]
    public void ThenTheResultShouldBe(string p0)
    {
        ScenarioContext.Current.Pending();
    }
}

}

I always enclose string values in '' rather then nothing or "" as this makes them easier to work with in c#, especially in the regex. I'm not sure what the expected behaviour should be if you are not specifying the value to use in your steps, they don't even read correctly to me as they seem like incomplete sentances.

If I omit the '' around the then I get the smae issue and maybe this is something that needs to be looked at, but the simple workaround is to always wrap strings which are parameters in ''.

If you file a bug (or comment on an existing one) on github, then I'll try and make some time to look at it, but it won't be soon.

Sam

On Tue, Aug 18, 2015 at 11:00 PM, wcdeich4 notifications@github.com wrote:

Feature: SpecFlowFeature1 string processing

@mytag Scenario Outline: String Parsing Given I have entered And Then I entered And Then I also entered When I press combine the strings Then the result should be

Examples:

| ValueA | ValueB | ValueC | ValueD | | Hello | Big | World | HelloBigWorld | | Goodbye | Small | World | GoodbyeSmallWorld | | http:// | plaza.ufl.edu/ | gospelsf | http://plaza.ufl.edu/gospelsf |

-----------------------------Generates------------------------------------------------------------

using System; using TechTalk.SpecFlow;

namespace SpecFlowTests { [Binding] public class SpecFlowFeature1Steps { [Given(@"I have entered Hello")] public void GivenIHaveEnteredHello() { ScenarioContext.Current.Pending(); }

[Given(@"Then I entered Big")]
public void GivenThenIEnteredBig()
{
    ScenarioContext.Current.Pending();
}

[Given(@"Then I also entered World")]
public void GivenThenIAlsoEnteredWorld()
{
    ScenarioContext.Current.Pending();
}

[When(@"I press combine the strings")]
public void WhenIPressCombineTheStrings()
{
    ScenarioContext.Current.Pending();
}

[Then(@"the result should be HelloBigWorld")]
public void ThenTheResultShouldBeHelloBigWorld()
{
    ScenarioContext.Current.Pending();
}

}

}

— Reply to this email directly or view it on GitHub https://github.com/techtalk/SpecFlow/issues/473#issuecomment-132368829.

[wcdeich4]

Thank you. Where would I file the bug? Perhaps it is a documentation issue since I never read any where that you should put ' or " around the value tags inside the sentences. I don't think they do it that way in Java Cucumber, but if it works this way in SpecFlow, then we should be sure it is well documented.

[samholder]

You can file the bug on the GitHub specflow page. Nothing in cucumber says you should use quotes I don't believe but it seems without them the generation is not perfect and could be improved but I think you have to include the part in the step. It doesn't make any sense otherwise. At least not to me. On 19 Aug 2015 15:54, "wcdeich4" notifications@github.com wrote:

Thank you. Where would I file the bug? Perhaps it is a documentation issue since I never read any where that you should put ' or " around the value tags inside the sentences. I don't think they do it that way in Java Cucumber, but if it works this way in SpecFlow, then we should be sure it is well documented.

— Reply to this email directly or view it on GitHub https://github.com/techtalk/SpecFlow/issues/473#issuecomment-132606131.

[wcdeich4]

Hi, I went to https://github.com/techtalk/SpecFlow but I don't see a link to file the bug.

In reference to Sam's last comment, I never tried to create a scenario outline without I just didn't know it was necessary to use ' or " in the Gherkin or the Example rows, that's why I said this could be a documentation problem - although, since ' or " is not needed in Java's Cucumber, it would at least be "nice to have" for that not to be necessary in C# SpecFlow either.

[merrua]

I was going to add an example for this to help out, but there actually already is one. It's just not that easy to search for.

Feature: US01 - Book Search Scenario: Title should be matched When I search for books by the phrase 'Domain' Then the list of found books should contain only: 'Domain Driven Design'

SearchSteps.cs [When(@"I search for books by the phrase '(.*)'")] public void WhenISearchForBooksByThePhrase(string searchTerm) { var controller = new CatalogController(); actionResult = controller.Search(searchTerm); } https://github.com/techtalk/SpecFlow-Examples/tree/master/ASP.NET-MVC/BookShop/BookShop.AcceptanceTests/StepDefinitions

scenario example of the above with a table
Scenario: Title should be matched tablexample
When I search for books by the phrase ' <Text>'
Then the list of found books should contain only: ' <TTitle>'
| TTitle                | Text   |
| Domain Driven Design | Domain |

[MNF]

samholder commented on Aug 19, 2015

If you file a bug (or comment on an existing one) on github, then I'll try and make some time to look at it, but it won't be soon.

Is this thread considered as a filed bug?
Or we need to create it in some other place?

I believe that every new user who follows cucumber documentation https://github.com/cucumber/cucumber/wiki/Scenario-outlines will try parameters delimetered by <>. And will have a problem in generation steps, until find the workaround that In SpecFlow it is recommended to add extra single quotation marks. Fix this will help to avoid confusion for first time users.

Example that I used:

Feature: test parameter with or without quotes

Scenario Outline: test parameter with or without quotes
When Booking Date <BookingDate> without quotes, extra parameter is generated
When Booking Date '<BookingDate> ' with quotes, correct parameter is generated
Examples:
  | BookingDate |
  | 11-05-2015  |

//However specFlow Generates 2 parameters and at runtime fails with cast "" to int.
        [ When( @"Booking Date (.*)(.*) without quotes, extra parameter is generated")]
        public void WhenBookingDateWithoutQuotesExtraParameterIsGenerated( string p0, int p1)
        {
            ScenarioContext.Current.Pending();
        }

        [ When( @"Booking Date '(.*)' with quotes, correct parameter is generated" )]
        public void WhenBookingDateWithQuotesCorrectParameterIsGenerated( string p0)
        {
            ScenarioContext.Current.Pending();
        }

[wcdeich4]

It works if you put double or single quotes inside the line of gherkin - but I could not find this in the documentation - so you could say it is a bug in the documentation - although it would be nice if double/single quotes were not required b/c Cucumber in Java does not require it

Date: Sun, 13 Mar 2016 01:10:46 -0800 From: notifications@github.com To: SpecFlow@noreply.github.com CC: wcdeich4@hotmail.com Subject: Re: [SpecFlow] Scenario Outline String Problem (#473)

samholder commented on Aug 19, 2015

If you file a bug (or comment on an existing one) on github, then I'll try and make some time to look at it, but it won't be soon.

Is this thread considered as a filed bug?

Or we need to create it in some other place?

I believe that every new user who follows cucumber documentation

https://github.com/cucumber/cucumber/wiki/Scenario-outlines will try

parameters delimetered by <>. And will have a problem in generation steps, until find the workaround that In SpecFlow it is recommended to add extra

single quotation marks.

Fix this will help to avoid confusion for first time users.

Example that I used:

Feature: test parameter with or without quotes

Scenario Outline: test parameter with or without quotes When Booking Date without quotes, extra parameter is generated When Booking Date ' ' with quotes, correct parameter is generated Examples: | BookingDate | | 11-05-2015 |

//However specFlow Generates 2 parameters and at runtime fails with cast "" to int. [ When( @"Booking Date (.)(.) without quotes, extra parameter is generated")] public void WhenBookingDateWithoutQuotesExtraParameterIsGenerated( string p0, int p1) { ScenarioContext.Current.Pending(); }

    [ When( @"Booking Date '(.*)' with quotes, correct parameter is generated" )]
    public void WhenBookingDateWithQuotesCorrectParameterIsGenerated( string p0)
    {
        ScenarioContext.Current.Pending();
    }

— Reply to this email directly or view it on GitHub.

[MNF]

@wcdeich4, incorrect generation without quotes I consider as a bug, because SpecFlow uses standard Gherkin language and even doesn't store its own version of documentation, but just refers to https://github.com/cucumber/cucumber/wiki/Scenario-outlines.

Documenting workaround is not sufficient. Note that SpecFlow wiki https://github.com/techtalk/SpecFlow/wiki/Using-Gherkin-Language-in-SpecFlow already describe the workaround

Placing single quotation marks (') around placeholders (eg. '')improves SpecFlow's ability to parse the scenario outline and generate more accurate regular expressions and test method signatures.

[wcdeich4]

@MNF Ok, well feel free to file a bug & adjust the code to be more flexible :smile: I'm busy & I'm not sure how to make that particular code change in SpecFlow, but if you want to fix it, that's great :smile:

[MNF]

@wcdeich4 @samholder, you suggested to file a bug. Where and how should I do it? Is https://github.com/techtalk/SpecFlow/issues/ the place for SpecFlow bugs? Does this thread itself considered as bug report?

[samholder]

yes this counts as a bug report. I was confused as I get emails from here and the google group to the same folder in my email.