What's the difference to Turnip?

[jmuheim]

I'm using Turnip at the moment. But its author doesn't have much time for it, so I fear it will become abandonware some time.

Can you tell me what's the difference to Turnip? Both look very similar. A quick search on the web showed that Spinach doesn't support passing values to steps. Is this true?

Thank you.

[oriolgual]

Yes Joshua, Spinach doesn't support passing values to steps nor regex in tis steps (this is not unintentional). They are actually quite similar, but we're using a subset of Gherkin. In Spinach, features are implemented in plain Ruby classes, and you can share steps (or any code) with other features like you would do normally in Ruby (either via modules or having objects collaborate).

Also, Spinach is test framework agnostic (you can use minitest, testunit, rspec, you name it).

[jmuheim]

Okay, thank you. What's the reason for your decision not to support passing values to steps?

[opsb]

I'm not sure what the maintainers would say but here's my personal take on it:

When you use regexes to define common steps the tendency is for people to slice and dice english into a set of steps that they can reuse. The problem is, english is incredibly expressive and you'd need natural language processing to support a natural style of language in your features. For example, take a look at this example from the pickle docs:

Given a user exists
And a post exists with author: the user

Given a person "fred" exists
And a person "ethel" exists
And a fatherhood exists with parent: person "fred", child: person "ethel"

This is basically a programming language that uses some english words. Compare this to

Given a user has created a post

Given fred is ethel's father

When you're writing spinach steps you're free to write english any way you like. This is in large part because steps aren't global, so you don't need to worry about accidentally triggering any that have been defined already.

TLDR, spinach's philosophy is that code re-use should be done in ruby, not in the mapping from english => ruby.

[jeffnyman]

Just adding my two cents to this: While I understand the notion that the maintainers are going for with not allowing regex, I think not providing options to people can be a bad approach to take. It very much depends on the complexity of the domain you are dealing with. I work in industries like clinical trials, banking, hedge funds, and ad serving. In those cases, when we create test specs, the ability to allow re-use at the English level is often extremely helpful. That being said, I can understand where people are coming from with this.

However, another area that Spinach does not follow Gherkin (currently) is in allowing scenario outlines, tables or extended (doc) strings. These are constructs that some feel are anti-patterns but, again, the choice really depends on the domain you are working in. To disallow these constructs really means you are not adhering to Gherkin. (And to be sure, Spinach did -- still does? -- use a modified form of Gherkin.)

In every place I have worked, when Spinach was considered relative to Turnip or Cucumber, Spinach always lost simply because it removed too many choices from people. That doesn't mean Spinach is a bad tool at all, but it does pay to be aware of the limitations that have been put in place due to an opinionated view about what a good "test spec" ("Feature File") should be. Those opinions are by no means "bad", but you should see if your opinions align.

[opsb]

@jnyman Could you give an example of a feature that you feel benefits from using regexes (perhaps one of the technical specs you mention)? I think it would really help the discussion if we could look at something concrete (I'm a big fan of the way spinach does things but I don't think the case is closed by any means).

[jeffnyman]

These are going to be disconnected from the wider specs, of course, including some narrative elements that explained parts of what the scenarios dealt with. With that in mind, here's a particular example:

Scenario: Apply Discount for Mid-Range, Long-Standing Games
  * a discount of 2% is applied when an Active customer purchases a mid-range, long-standing game

That can map down to:

(/^a discount of (/d+)% is applied when an (.*) customer purchases a (.*) game$/)

That was actually done for Valve as part of their Steam product. This is the way the customer wanted to write some of their scenarios and the variations were exactly the regex points. Now, that was, as you can see, very "generic" in that it called into question what specific "mid-range, long-standing game" would be used. Another scenario we used:

Scenario: Apply Discount for Mid-Range, Long-Standing Games
  GIVEN an Active customer
  WHEN  the customer purchases the game "Midsummer Evening of Walking Living Dead"
  THEN  a discount of %2 is applied

So here the particular game is used but, of course, the matcher would have a placeholder that allowed the phrase to be reused for any game. Likewise, the "discount of %2" was parameterized.

Note: I do realize someone could say these tests should be pushed to unit tests. However, many clients do not want to do that. They like having tests just like these that express their business intent as test cases with specific examples.

Here's a slightly more complicated example from the clinical trial industry:

Scenario: CRO Customer, Built-In Resources
  GIVEN a CRO customer with a standard plan that has a CRO service provider
  AND a CRO service provider called "TestCRO" that has the following:
     | a published any sponsor billing rate card |
     | a published deleted sponsor-specific billing rate card |
  WHEN you replace the plan's original CRO service provider with TestCRO
  THEN the rates for each of the built-in resources for a location match the any sponsor billing rate card

Here's an example where they really liked having a table with specific conditions. The WHEN and the THEN were actually literal phrases, not parameterized. But the AND was parameterized as was the GIVEN and they didn't want to have to different matchers for every single permutation. Another one from that same industry looked like this:

GIVEN a plan where:
  | Trial subjects are set to 10 |
  |  Fees and Costs are set to 0 |
  | Subject rate cost is set to 1000 |
WHEN the data collection method is "Empty"
THEN Data Capture Costs are zero.

This was necessary because companies like clinical trial providers or banks or insurance companies are audited. If the above test specs are used as part of that audit (and they are), that is exactly what people want to see. But as you can imagine, phrases like "Trial subjects are set to", "Fees and costs are set to" and so on are parameterized.

These are just some examples. Even that I sort of presented these as a fait accompli, I'm certainly open to discussion on them. I hope these are relevant enough and significant enough to give you some idea of what I'm talking about.

[woahdae]

@jnyman +1! I've had the same assessment that Spinach threw out features I really liked from Cucumber.

I use Ruby because it entrusts powerful concepts to professionals for using to powerful effect, but I don't use Cucumber because global regex dispatch is just the worst thing I could come up with. The tools it provides are great though, and (IMO, obviously) without ex. tables and step parameters, I'd rather just implement well-worded rspec blocks for acceptance testing and show stakeholders the verbose output. It's not drastically different than spinach at that point, and a lot simpler.

Again, with tables and step parameters, the value proposition totally changes.

[opsb]

@jnyman only just saw your post. From the examples you've shown you only specify a particular value once (e.g. a discount of 2%), is your point that you would use the same step with many different values?

[jeffnyman]

Correct, yes. In those cases: same step, different values. However, we also parameterized steps that didn't necessarily look as if they were. That's what this bit was showing:

(/^a discount of (/d+)% is applied when an (.*) customer purchases a (.*) game$/)

As another example:

WHEN you replace the plan's original CRO service provider with TestCRO

Was actually behind the scenes something like this:

(/^you replace the plan's (.*) service provider with (.*)$/)

Maybe not parameterized quite that simply, but you get the idea. We could have done that with placeholders (a la Turnip), such as:

step (you replace the plan's :cro_type service provider with :cro_name)

Either way, we could still have a parameterized step.

This made it easier for us to train testers on what steps to use and cut down on the number of steps we needed. Granted, though, there are the issues that people usually bring up which is that it can be unclear what values a step can take if its regex'd or given placeholders.

That said, I've found clients prefer the flexibility of being able to use whatever solution they like, even if there is the potential for shooting themselves in the foot.

This also allowed us to utilize a data builder pattern, wherein certain "allowed phrases" were stored in a data building file. If a step didn't use an approved phrase, we could signal that to the tester but without having the requirement of creating another step definition necessarily. All they had to do was add a line to the data builder, and our parameterized steps could handle it.

Along with all this, sometimes tables were simply found to be a good solution for expressing how business logic worked. (This particularly was the case in the banking and hedge fund industry, for example.)

If there was one part of the Gherkin pantheon that was the least used in my experiences, it tended to be the scenario outline.

[opsb]

This made it easier for us to train testers on what steps to use and cut down on the number of steps we needed

Perhaps this is a philosophical divide but this is the exact reason that I don't like parameterized steps. What you end up with is a pseudo language that leads to scenarios which have awkward english (certainly in my experience anyway).

By the sound of it you have testers writing scenarios using existing steps. This is something I've never considered before as I've always been provided with the scenarios by stakeholders or written them myself. In my opinion this requirement makes spinach pretty unsuitable for you. The philosophy behind spinach is that you use the full range of english when writing each scenario. This results in far more steps with duplication handled far more at the ruby level rather than the english level. Obviously you need more involvement from developers to achieve this though so they would probably become a bottleneck for your testers.

[jeffnyman]

Indeed, good points. Yes, the domains I work in tend to rely on people creating a ubiquitous language but one that is still constrained. Ultimately these tools are (to me) testing tools, if you consider that testing is a design activity predicated upon collaboration. So, in my environments, we don't want people expressing things in English any way they want; rather, we want people settling on a relatively constrained way to describe and discuss their business domain.

This allows the business to work with testers to express requirements (which these things really are) as test conditions and data conditions.

This allows us to build in mechanisms that tell us if people are using phrases they should not. So for example, consider these phrases:

Given a study
Given a phase I study
Given a study with any phase
Given a phase III study
Given a late stage study

Of those, the second and third were not valid. The reason is that, with the second, there were actually two types of "phase I" study so the above was too vague. With the third, the issue was that you could not use "any phase" because cost drivers were too specific to phase type. The point there being is that each of those steps was a single step definition so we could use a data builder pattern to determine if an invalid phrase was being used.

We could have had individual step definitions for each, of course, but then we had to account for how someone might say "phase III." They might do "phase three", "phase 3", etc, with each being a different phrase. Instead we have a parameterized step that connects to a data builder and, while doing so, could check for invalid conditions at the spec level.

As you say, it's perhaps a philosophical divide and quite an interesting one, really. If nothing else, this definitely shows perhaps the different ways people will want to use these tools. I appreciate your insight.

[ywen]

Can we close this issue since there is no action needed?