step matching is not deterministic.

[dnozay]

in lettuce.core.Step:

    def _get_match(self, ignore_case):
        matched, func = None, lambda: None

        for regex, func in STEP_REGISTRY.items():
            matched = re.search(regex, self.sentence, ignore_case and re.I or 0)
            if matched:
                break

This makes the tests fail randomly (as in not 100% of the time):

======================================================================
FAIL: A feature with background should print it accordingly under verbosity 3
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/damien/Work/github/lettuce/.tox/py27/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Users/damien/Work/github/lettuce/tests/functional/test_runner.py", line 1283, in test_output_background_with_success_colorless
    .format(line=line+2)  # increment is line number of step past line
  File "/Users/damien/Work/github/lettuce/tests/asserts.py", line 107, in assert_stdout_lines
    assert_lines(sys.stdout.getvalue(), other)
  File "/Users/damien/Work/github/lettuce/tests/asserts.py", line 43, in assert_lines
    assert_lines_unicode(original, expected)
  File "/Users/damien/Work/github/lettuce/tests/asserts.py", line 65, in assert_lines_unicode
    raise AssertionError(repr(msg.format(diff, original, expected)).replace(r'\n', '\n'))
AssertionError: 'Output differed as follows:

  Feature: Simple and successful                # tests/functional/bg_features/simple/simple.feature:1
    As the Lettuce maintainer                   # tests/functional/bg_features/simple/simple.feature:2
    In order to make sure the output is pretty  # tests/functional/bg_features/simple/simple.feature:3
    I want to automate its test                 # tests/functional/bg_features/simple/simple.feature:4

    Background:
      Given the variable "X" holds 2            # tests/functional/test_runner.py:1258

    Scenario: multiplication changing the value # tests/functional/bg_features/simple/simple.feature:9
-     Given the variable "X" is equal to 2      # tests/functional/bg_features/simple/steps.py:1258
?                                                                                              -- -
+     Given the variable "X" is equal to 2      # tests/functional/bg_features/simple/steps.py:5

  1 feature (1 passed)
  1 scenario (1 passed)
  1 step (1 passed)

Output was:

Feature: Simple and successful                # tests/functional/bg_features/simple/simple.feature:1
  As the Lettuce maintainer                   # tests/functional/bg_features/simple/simple.feature:2
  In order to make sure the output is pretty  # tests/functional/bg_features/simple/simple.feature:3
  I want to automate its test                 # tests/functional/bg_features/simple/simple.feature:4

  Background:
    Given the variable "X" holds 2            # tests/functional/test_runner.py:1258

  Scenario: multiplication changing the value # tests/functional/bg_features/simple/simple.feature:9
    Given the variable "X" is equal to 2      # tests/functional/bg_features/simple/steps.py:5

1 feature (1 passed)
1 scenario (1 passed)
1 step (1 passed)

Expected was:

Feature: Simple and successful                # tests/functional/bg_features/simple/simple.feature:1
  As the Lettuce maintainer                   # tests/functional/bg_features/simple/simple.feature:2
  In order to make sure the output is pretty  # tests/functional/bg_features/simple/simple.feature:3
  I want to automate its test                 # tests/functional/bg_features/simple/simple.feature:4

  Background:
    Given the variable "X" holds 2            # tests/functional/test_runner.py:1258

  Scenario: multiplication changing the value # tests/functional/bg_features/simple/simple.feature:9
    Given the variable "X" is equal to 2      # tests/functional/bg_features/simple/steps.py:1258

1 feature (1 passed)
1 scenario (1 passed)
1 step (1 passed)
'

This is because there are 2 steps registered with different regexes but that would match the same pattern.

e.g. in test_runner.py

    line = currentframe().f_lineno  # get line number
    @step(ur'the variable "(\w+)" holds (\d+)')
    @step(ur'the variable "(\w+)" is equal to (\d+)')
    def just_pass(step, *args):
        pass

and in tests/functional/bg_features/simple/steps.py

@step(u'Given the variable "([^"]*)" is equal to 2')
def given_the_variable_group1_is_equal_to_2(step, group1):
    pass

[dnozay]

solutions:

• use a SortedDict, or
• use another datastructure which preserves order, or
• use unordered dict and raise an exception if multiple matches are found (e.g. MultipleObjectsReturned)

[qrilka]

Stumbled upon similar problem with @given. And 2.5 years passed since this ticket was created :)