Tyler-Keith-Thompson / CucumberSwift

A lightweight swift Cucumber implementation
https://tyler-keith-thompson.github.io/CucumberSwift/documentation/cucumberswift/
MIT License
74 stars 19 forks source link

Wrong order of hooks execution. #80

Closed Hsilgos closed 1 year ago

Hsilgos commented 1 year ago

In current implementation hooks are called in the order in "whatever order they appear in the code" unless priority is specified.

This is inconsistent with other's implementations like cucumber-jvm. Documentation for @Before in cucumber-jvm:

    /**
     * @return the order in which this hook should run. Lower numbers are run first.
     * The default order is 10000.
     */
    int order() default 10000;

Documentation for @After in cucumber-jvm:

    /**
     * @return the order in which this hook should run. Higher numbers are run first.
     * The default order is 10000.
     */
    int order() default 10000;

As you see order has opposite meaning for @Before and for @After. Also articles say that order of execution is reversed for "@After" hooks, for example https://www.axelerant.com/blog/how-work-cucumber-hooks#H3140 For the same priority (or for absence of priority) @After is called in reversed to registration order.

So basically for code

extension Cucumber: StepImplementation {
    public func setupSteps() {
        BeforeScenario(priority: 1) { scenario in
          // Before 1.1
        }

        AfterScenario(priority: 1) { scenario in
           // After 1.1
        }

        BeforeScenario(priority: 1) { scenario in
          // Before 1.2
        }

        AfterScenario(priority: 1) { scenario in
           // After 1.2
        }

        BeforeScenario(priority: 2) { scenario in
           // Before 2
        }

        AfterScenario(priority: 2) { scenario in
           // After 2
        }
    }
}

The order must be

Before 1.1 -> Before 1.2 -> Before 2 -> Scenario -> After 2 -> After 1.2-> After 1.1

Pay attention that After hook with higher priority is called first and After hook with the same priority are called in reversed (to declaration sequence) order.

Why is it important?

I build more complex logic where steps are grouped by functionality, so it rather look like:

// Foo depends on Bar and I want to init Bar before Foo and deinit Bar after Foo
class Foo {
  let bar: Bar
  public func registerSteps() {
     BeforeScenario() { scenario in
     }

    AfterScenario() { scenario in
       self.bar.doSomething();
    }
  }
}

class Bar {
  public func registerSteps() {
     BeforeScenario() { scenario in
     }

    AfterScenario() { scenario in
    }
  }
}

extension Cucumber: StepImplementation {
    let bar = Bar()
    let foo = Foo()
    public func setupSteps() {
        foo.bar = bar
        bar.registerSteps()
        foo.registerSteps()
    }
}

As I told I want the order Bar.BeforeScenario -> Foo.BeforeScenario -> Scenario -> Foo.AfterScenario -> Bar.AfterScenario In this simple example I can use parameter order, but when I have many such classes in many files it became problematic and error prune to specify order everywhere.

Necessary change looks easy, but breaks existing code. How about to introduce an option like continueTestingAfterFailure which changes this behaviour? reverseOrderForAfterHooks?

Tyler-Keith-Thompson commented 1 year ago

I do like the idea of adding configuration for this. CucumberSwift configuration is an inconsistent mix of Swift and plist values.

How would you feel about adding a defaulted variable to the StepImplementation protocol that contained at least this and perhaps other values? They'd have to still work as they are, but it'd give us a place to add new things from now on.

So, something like a CucumberSwift.Config type