sebaslogen / CleanGUITestArchitecture

Sample project of Android GUI test automation using Espresso, Cucumber and the Page Object Pattern
MIT License
137 stars 31 forks source link

Undefined step(s) returned when executing feature file #17

Closed willpowr closed 7 years ago

willpowr commented 7 years ago

Hi,

I've loaded, compiled and run the application successfully on an Android emulator. However when I run the feature file - Feature: defaults, the test fails with the output below.

The tests run successfully when the step definition class StepDefinitions is executed directly via the IDE context menu.

Any ideas on what's going wrong? If you require additional output files I'll be happy to provide them.

Environment: Windows 10 Android Studio 2.3.2

Thanks,

H

... Testing started at 12:28 ...

Undefined step: Given I see the login page

Undefined step: When I login with user name "Sebas" and password "passion"

Undefined step: Then I see the welcome page

Undefined step: And the title is "Welcome Sebas"

Undefined step: Given I see the login page

Undefined step: When I login with user name "sebaslogen" and password "passion"

Undefined step: Then I see the welcome page

Undefined step: And the title is "Welcome sebaslogen"

Undefined step: Given I see the login page

Undefined step: When I login with user name "ThirdUser" and password "passion"

Undefined step: Then I see the welcome page

Undefined step: And the title is "Welcome ThirdUser"

3 Scenarios (3 undefined) 12 Steps (12 undefined) 0m0.000s

You can implement missing steps with the snippets below:

@Given(...

sebaslogen commented 7 years ago

Hi, Can you explain in more detail how are trying and failing to run the tests?

You can run tests from the IDE and from the command line but the execution is not filtered per file, but rather by tags (all tests are executed if no tag is given as filter). For more info see https://github.com/sebaslogen/CleanGUITestArchitecture/issues/15

willpowr commented 7 years ago

I attempted the following steps...

  1. Select the Android view in the project pane.
  2. Expand the folder app/assets/features(androidTest)
  3. Right-click login.feature

image

sebaslogen commented 7 years ago

Indeed, that's not possible because it's not how Cucumber recognizes tests. Tests are recognized by tags rather than files or folders.

In addition, tags can be added before a "Feature: ..." line, not only before a "Scenario: ..." line. Example:

@login-feature
Feature: Login in the application

  @ScenarioId("FUNCTIONAL.AUTH.SCN.001") @UserStory("MyApp-135") @login-scenarios
  Scenario: User can login with valid user name and password
    Given I see the login page
    When I login with user name "Sebas" and password "passion"
    Then I see the welcome page
    And the title is "Welcome Sebas"

  @ScenarioId("FUNCTIONAL.AUTH.SCN.002") @UserStory("MyApp-135") @login-scenarios
  Scenario: User can login with valid second user name and password
    Given I see the login page
    When I login with user name "sebaslogen" and password "passion"
    Then I see the welcome page
    And the title is "Welcome sebaslogen"
willpowr commented 7 years ago

The examples given pertain to running the test from the command line which works as expected in my environment. I guess my question is, how is it possible to run cucumber feature tests using Android Studio GUI using the Run window / toolbar run buttons? Can tags be used somewhere in the Run/Debug Configuration?

willpowr commented 7 years ago

...The intellij documentation instructions seem to suggest right-clicking on feature files is doable. https://www.jetbrains.com/help/idea/running-cucumber-tests.html

Running Cucumber Tests

The Cucumber feature files are run same way as the other executables, with certain run/debug configuration settings.

Prior to running a test, you have to set up run/debug configuration for a particular feature file or scenario, or for the whole bunch of features within a directory. Creating a run/debug configuration is described in the section Creating and Editing Run/Debug Configurations.

However, there is always the possibility to run Cucumber tests with the default settings, which is described below.

Results of tests execution are displayed in the Test Runner tab of the Run tool window.

To run all feature files in a directory

In the Project tool window, right-click the directory where the feature files are stored. On the context menu of the directory, choose Run all features in .

Does Android Studio not inherit this functionality?

sebaslogen commented 7 years ago

In Android Studio, run connectedAndroidTest in the right Gradle tab and then edit the run configuration to add under the Script parameters something like -Ptags="@login-scenarios,@kitkat"

Let me know if it works.

As for the IntelliJ feature I have no info.

willpowr commented 7 years ago

Thanks for your time on this. I'm sure it will help 100s of people! I ran the connectedAndroidTest as advised which completed successfully. I edited the tags in the scenarios of the feature file to @login-scenario1, @login-scenario2, @login-scenario3 I added-Ptags="@login-scenario2" to Script parameters and re-ran the script.

SUCCESS! Only the 2nd scenario was executed.

The question remains however... Does Android Studio allow the more user friendly and visually appealing execution of feature files as quoted above?

ArunaBalasubramanian commented 7 years ago

Hello, The connectedAndroidTest with script parameters doesn't seem to filter my scenarios based on tags. Should the build.gradle file be instructed about the usage of -Ptags parameter?

My cucumber runner looks like below, I do not use a CucumberTestCase file that has other added properties.

@CucumberOptions(features = "features") public class CucumberInstrumentationRunner extends MonitoringInstrumentation { private final CucumberInstrumentationCore instrumentationCore = new CucumberInstrumentationCore(this);

@Override
public void onCreate(final Bundle bundle) {
    super.onCreate(bundle);
    instrumentationCore.create(bundle);
    start();
}

@Override
public void onStart() {
    waitForIdleSync();
    instrumentationCore.start();
}

}

sebaslogen commented 7 years ago

@ArunaBalasubramanian the -Ptags is already handled in the build.gradle here, so I'm not sure what is the difference between your configuration and what @willpowr has already running.

Maybe try restoring the CucumberTestCase class and see if that helps.

ArunaBalasubramanian commented 7 years ago

Thanks @sebaslogen for your response.

build.zip

This is my build.gradle. Do you see anything that needs change? Adding a CucumberTestCase is throwing me "cucumber.runtime.CucumberException: No CucumberOptions annotation". I tried solving using https://github.com/sebaslogen/CleanGUITestArchitecture/issues/3, but didn't help.

sebaslogen commented 7 years ago

@ArunaBalasubramanian I don't see anything wrong with your build.gradle

As for the CucumberTestCase the easiest way is to copy the files and structure under androidTest folder of this sample project, start by reducing the feature and steps to a simple step opening the app and then get it running.

ArunaBalasubramanian commented 7 years ago

Thanks @sebaslogen for the insight. I had to tweak a little on the project structure and finally it did work.

amesandy-zz commented 6 years ago

how do you add the features(android test) folder on assets directory?

sebaslogen commented 6 years ago

@amesandy you can create the folder structure using the file explorer of your OS. Just follow the same path structure as the code in this repository