search

mobile-dev-inc / maestro

Painless Mobile UI Automation
https://maestro.mobile.dev/
Apache License 2.0
5.89k stars 281 forks source link

"Could not find flows needed for execution in order" when including & excluding flows that are defined in an execution order #1681

Open DavidREntwistle opened 9 months ago

DavidREntwistle commented 9 months ago

Describe the bug If you include tags and exclude tags as well as having flows defined in the execution order of the config.yaml, then the java.lang.IllegalStateException: Could not find flows needed for execution in order is thrown:

java.lang.IllegalStateException: Could not find flows needed for execution in order: flow_b, flow_c
    at maestro.orchestra.workspace.ExecutionOrderPlanner.getFlowsToRunInSequence(ExecutionOrderPlanner.kt:21)
    at maestro.orchestra.workspace.WorkspaceExecutionPlanner.plan(WorkspaceExecutionPlanner.kt:103)
    at maestro.cli.command.TestCommand.call(TestCommand.kt:122)
    at maestro.cli.command.TestCommand.call(TestCommand.kt:46)
    at picocli.CommandLine.executeUserObject(CommandLine.java:1933)
    at picocli.CommandLine.access$1200(CommandLine.java:145)
    at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2332)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2326)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2291)
    at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2159)
    at maestro.cli.DisableAnsiMixin$Companion.executionStrategy(DisableAnsiMixin.kt:22)
    at picocli.CommandLine.execute(CommandLine.java:2058)
    at maestro.cli.AppKt.main(App.kt:117)

To Reproduce Flow set-up: testFolder/ flow_a.yaml > tags: - includeTest flow_b.yaml > tags: - excludeTest flow_c.yaml > tags: - excludeTest flow_d.yaml > tags: - includeTest

config set-up:
# NOTE: No Global Tags
executionOrder:
  continueOnFailure: true
  flowsOrder:
    - flow_a
    - flow_b
    - flow_c
    - flow_d
  1. Start an android emulator
  2. Run maestro test --include-tags=includeTest --exclude-tags=excludeTest testFolder/

Expected behavior TBC - However, I would expect if we are running against include & exclude tags, that we do not face an exception with the execution order because we have excluded a flow that is also defined in the sequential ordering.

Environment information (please complete the following information):

Additional context If we want to keep an execution order e.g. flow_a > flow_b > flow_c > flow_d but run against different including & excluding tags, I would expect that we would maintain the execution order but omit flows from this execution order that have been excluded with the exclude tag e.g. flow_a > flow_b > flow_c > flow_d.

hasnain1997 commented 4 months ago

we are facing similar issue where execution flows and tags are conflicting Lets say I have 4 test cases:

test1 (I have tagged it as [payment])
test2 (I have tagged it as [payment])
test3 (I have tagged it as [checkout])
test4 (I have tagged it as [checkout])

in my config.yaml I have the following execution order

  flowsOrder:
    - test1
    - test2
    - test3
    - test4

now when I try to run maestro test --format junit --include-tags=checkout tests/ I get the following error java.lang.IllegalStateException: Could not find flows needed for execution in order: test1, test2 as they are not part ofcheckout tag.

can you guys please look into this issue.

bartekpacia commented 4 months ago

See also discussion in community slack