techcobweb commented 1 month ago

Bug

When I run the following scerario I get a null pointer exception.

Feature: Browse the catalog and order

  Scenario Outline: Browse a Product from catalog and place an order

    Given a terminal
    Then wait for "CICS54" in any terminal field
    And type "CICS54" on terminal
    And press terminal key ENTER
    And wait for terminal keyboard
    Then wait for "Signon to CICS" in any terminal field

    And wait for terminal keyboard

        And type <username> on terminal
        And press terminal key TAB
        And type <password> on terminal

      And press terminal key ENTER

        And wait for terminal keyboard
        Then check "Sign-on is complete" appears only once on terminal      

    Examples:
        | username | password |
        | X123455  | YXXXXXXX |

Run log looks like this:

16/05/2024 14:11:15.745 DEBUG d.g.f.FrameworkInitialisation - galasa home used to determine CPS location.
16/05/2024 14:11:15.746 DEBUG d.g.f.FrameworkInitialisation - Configuration Property Store is file:///C:/Users/0015NA744/.galasa/cps.properties
16/05/2024 14:11:15.776 DEBUG d.g.f.FrameworkInitialisation - Selected CPS Service is dev.galasa.framework.internal.cps.FpfConfigurationPropertyStore
16/05/2024 14:11:15.798 DEBUG d.g.f.FrameworkInitialisation - Dynamic Status Store is file:///C:/Users/0015NA744/.galasa/dss.properties
16/05/2024 14:11:15.799 DEBUG d.g.f.FrameworkInitialisation - Obtained value of DSS from galasa home
16/05/2024 14:11:15.827 INFO  d.g.f.FrameworkInitialisation - Submitting test file:/C:/Users/0015NA744/eclipse-workspace/dev.galasa.zos.banking/dev.galasa.zos.banking.account/src/main/java/dev/galasa/zos/banking/account/ZuesLogonTest.feature
16/05/2024 14:11:15.969 INFO  d.g.f.FrameworkInitialisation - Allocated Run Name U281 to this run
16/05/2024 14:11:15.969 INFO  d.g.f.FrameworkInitialisation - Run name is U281
16/05/2024 14:11:15.975 DEBUG d.g.f.FrameworkInitialisation - Adding Result Archive Store location file:///C:/Users/0015NA744/.galasa/ras/
16/05/2024 14:11:15.977 DEBUG d.g.f.FrameworkInitialisation - Result Archive Stores are [file:///C:/Users/0015NA744/.galasa/ras/]
16/05/2024 14:11:16.065 DEBUG d.g.f.FrameworkInitialisation - Credentials Store is file:///C:/Users/0015NA744/.galasa/credentials.properties
16/05/2024 14:11:16.077 INFO  d.g.f.FrameworkInitialisation - Framework initialised
16/05/2024 14:11:16.121 ERROR dev.galasa.boot.Launcher.launch - Unable to run test class
dev.galasa.boot.LauncherException: java.lang.NullPointerException
    at dev.galasa.boot.felix.FelixFramework.runTest(FelixFramework.java:238)
    at dev.galasa.boot.Launcher.launch(Launcher.java:166)
    at dev.galasa.boot.Launcher.main(Launcher.java:120)
Caused by: java.lang.NullPointerException
    at dev.galasa.framework.spi.language.gherkin.GherkinTest.<init>(GherkinTest.java:141)
    at dev.galasa.framework.GherkinTestRunner.runTest(GherkinTestRunner.java:111)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at dev.galasa.boot.felix.FelixFramework.runTest(FelixFramework.java:236)
    ... 2 more
16/05/2024 14:11:16.123 DEBUG dev.galasa.boot.felix.FelixFramework.stopFramework - Stopping Felix framework
16/05/2024 14:11:16.134 INFO  d.g.f.Framework - Framework service deactivated
16/05/2024 14:11:16.179 DEBUG dev.galasa.boot.felix.FelixFramework.stopFramework - Felix framework stopped
16/05/2024 14:11:16.295 ERROR dev.galasa.boot.Launcher.main - Exiting launcher due to exception
dev.galasa.boot.LauncherException: java.lang.NullPointerException
    at dev.galasa.boot.felix.FelixFramework.runTest(FelixFramework.java:238)
    at dev.galasa.boot.Launcher.launch(Launcher.java:166)
    at dev.galasa.boot.Launcher.main(Launcher.java:120)
Caused by: java.lang.NullPointerException
    at dev.galasa.framework.spi.language.gherkin.GherkinTest.<init>(GherkinTest.java:141)
    at dev.galasa.framework.GherkinTestRunner.runTest(GherkinTestRunner.java:111)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at dev.galasa.boot.felix.FelixFramework.runTest(FelixFramework.java:236)
    ... 2 more

hobbit1983 commented 1 month ago

Just run this locally (while sat in hospital) and was able to get it to work. My feature file looks like this

Feature: GherkinLog
  Scenario: Log Example Statement

    THEN Write to log "This is a log statement <Item>"

  Scenario: Log Statement Test

    THEN Write to log "This is a second scenario <Value>"

Examples:
    | Item | Value |
    | password | data |
    | password2 | data2 |

examining the difference between the reported feature file

hobbit1983 commented 1 month ago

So the main issue here is that the user is using Scenario Outline: and not Scenario:. We don't support that currently. Although we can update the code that will be in the next release - changing scenario as outlined above should be an acceptable workaround

techcobweb commented 1 month ago

Pull request here: https://github.com/galasa-dev/framework/pull/544

techcobweb commented 1 month ago

[x] Added Scenario Outline:
[x] Scenario:should not contain an Examples: section and vice versa

techcobweb commented 4 weeks ago

This story is turning out to be harder to implement than expected.

techcobweb commented 4 weeks ago

@hobbit1983 I tried the feature text you added above.

I got this in the trace:

----------------------- ****************************************************************************************************
----------------------- *** Start of test method GherkinLog#Log Example Statement
----------------------- ****************************************************************************************************
2024/05/28 17:24:42 JVM output: (runid:L1) : 28/05/2024 17:24:42.798 INFO  d.g.f.s.l.g.GherkinMethod - Executing Statement: THEN Write to log "This is a log statement <Item>"
2024/05/28 17:24:42 JVM output: (runid:L1) : 28/05/2024 17:24:42.798 INFO  d.g.c.m.i.g.CoreStatementOwner - This is a log statement password
2024/05/28 17:24:42 JVM output: (runid:L1) : 28/05/2024 17:24:42.804 INFO  d.g.f.s.l.g.GherkinMethod - Ending
----------------------- ****************************************************************************************************
----------------------- *** Passed - Test method GherkinLog#Log Example Statement
----------------------- ****************************************************************************************************
2024/05/28 17:24:42 JVM output: (runid:L1) : 28/05/2024 17:24:42.805 INFO  d.g.f.s.l.g.GherkinMethod - Starting
----------------------- ****************************************************************************************************
----------------------- *** Start of test method GherkinLog#Log Statement Test
----------------------- ****************************************************************************************************
2024/05/28 17:24:42 JVM output: (runid:L1) : 28/05/2024 17:24:42.805 INFO  d.g.f.s.l.g.GherkinMethod - Executing Statement: THEN Write to log "This is a second scenario <Value>"
2024/05/28 17:24:42 JVM output: (runid:L1) : 28/05/2024 17:24:42.805 INFO  d.g.c.m.i.g.CoreStatementOwner - This is a second scenario data
2024/05/28 17:24:42 JVM output: (runid:L1) : 28/05/2024 17:24:42.805 INFO  d.g.f.s.l.g.GherkinMethod - Ending
----------------------- ****************************************************************************************************
----------------------- *** Passed - Test method GherkinLog#Log Statement Test
----------------------- ****************************************************************************************************
2024/05/28 17:24:42 JVM output: (runid:L1) : 28/05/2024 17:24:42.805 INFO  d.g.f.s.l.g.GherkinTest - Ending
----------------------- ****************************************************************************************************
----------------------- *** Passed - Test class GherkinLog
----------------------- ****************************************************************************************************

i.e: The <Item> and <Value> parts of the feature are kind of being substituted, but there are several problems with this:

It's not valid gherkin syntax. Data tables should only exist within a Scenario Outline.
The substitution of the variables should work without the quotes.
The scope of the example should be with the scenario outline immediately above the example data, not shared with several scenarios.
Only one test is run, there should be two, as there are two lines in the data table.

techcobweb commented 4 weeks ago

I think this is the syntax we need to support (in BNF):

1a. <feature> := FEATURE_START <scenarioPartList> END_OF_FILE
2b. <scenarioPartList> := null
2c.                     | <ScenarioPart> <scenarioPartList>
3a. <scenarioPart> := <scenarioOutline>
3b.                 | <scenario>
4a. <scenario> := SCENARIO_START <stepList>
5a. <scenarioOutline> := SCENARIO_OUTLINE_START <stepList> EXAMPLES_START <dataTable>
6a. <stepList> := null
6b.             | STEP <stepList>
7a. <dataTable> := <dataHeaderLine> <dataValuesLineList>
8a. <dataHeaderLine> := DATA_LINE
9a. <dataValuesLineList> := DATA_LINE
9b.                       | DATA_LINE <dataValuesLineList>

galasa-dev / projectmanagement

Gherkin null pointer exception when using Examples table #1858

Bug