radish-bdd / radish

Behavior Driven Development tooling for Python. The root from red to green.
https://radish-bdd.github.io
MIT License
182 stars 49 forks source link

Structure of step object changes when using a scenario loop #350

Closed fudomunro closed 5 years ago

fudomunro commented 5 years ago

I sometimes use Scenario Loop as a way to test for flakiness.

I've discovered that when I do this, the step object has a different structure. Specifically, we usually use step.parent.parent to access the feature, but when running in a scenario loop, we need step.parent.parent.parent. We store some test data on the feature, so this is inconvenient.

Ideally, the step object would be the same, regardless of how the step was being run, but if the additional parent is needed, then I suggest:

  1. Note the difference in the Scenario Loop documentation.
  2. Create a way to access the feature more directly, such as step.feature.
fliiiix commented 5 years ago

So I created a branch with a shitty test for it: https://github.com/radish-bdd/radish/commit/9d64e5aab27e0b877b3afee2e6faa2d7bb4f8635 (can be run like this: pytest -s tests/unit/test_scenarioloop.py::test_scenarioloop_access_feature )

The 'problem' is that a step in a ScenarioLoop is nested inside a a IterationScenario. As far as I can tell this is by design. But I guess @timofurrer knows more here.

# a normal scenario
<radish.scenario.Scenario object at 0x7fd8ca2230f0>
Feature: I am a feature from foo.feature:1

# a loop scenario
<radish.iterationscenario.IterationScenario object at 0x7fd8c9eecbe0>
<radish.scenarioloop.ScenarioLoop object at 0x7fd8c9ecb240>
Feature: I am a feature from foo.feature:1

Also it's not clear to me what your use case exactly is. Why do you have references to a step and then need data from a feature. Can you share some example code?

timofurrer commented 5 years ago

The 'problem' is that a step in a ScenarioLoop is nested inside a a IterationScenario. As far as I can tell this is by design.

That's exactly right. The reason behind this is, that a ScenarioLoop basically consists of multiple virtual Scenariosfor each loop iteration calledIterationScenario`. Those virtual iteration scenarios* are expanded and created during parse time and are needed to keep track the context during a run and their results. That's also the case for an ExampleScenario for an ScenarioOutline.

I guess it makes sense to add a property step.feature to access the feature within a step.

timofurrer commented 5 years ago

I've implemented a feature property in the step object. It'll be released in v1.0.0