Fatal error in testjob

[tooh]

ENVIRONMENT:

-enabled skip-certificate-check

Started by user anonymous [EnvInject] - Loading node environment variables. Building on master in workspace /Users/peter/.jenkins/workspace/TestThemAll - Plugin Connecting to Testopia to retrieve automated test cases. Executing single build steps [TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson8869976021520379580.sh

• echo test test Executing iterative build steps [TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson2953766809693164667.sh
• echo test test FATAL: null java.lang.NullPointerException at jenkins.plugins.testthemall.Utils.createEnvironmentVariables(Utils.java:62) at jenkins.plugins.testthemall.Utils.buildTestCaseEnvVars(Utils.java:51) at jenkins.plugins.testthemall.TestopiaBuilder.executeIterativeBuildSteps(TestopiaBuilder.java:243) at jenkins.plugins.testthemall.TestopiaBuilder.perform(TestopiaBuilder.java:187) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717) at hudson.model.Build$BuildExecution.build(Build.java:199) at hudson.model.Build$BuildExecution.doRun(Build.java:160) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1488) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:236)

[kinow]

Either the testcase object is null, or the testcase ID is null. I thought both cases would never happen :-S

I'm sending a new version of the plug-in to your e-mail. This version adds logging. You'll have to enable logging in your jenkins instance - https://wiki.jenkins-ci.org/display/JENKINS/Logging :-)

[tooh]

ENVIRONMENT

• OSX Mountain Lion Server 2.0
• 1.0-SNAPSHOT (private-07/25/2012 20:36-kinow) -.Enabled logging

LOGFILE

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: Executing iterative build step

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: TestCase: id[2], script[]

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: Executing iterative build step

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: TestCase: id[2], script[./runjython.sh t/testFinder.py]

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: Executing iterative build step

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: TestCase: id[2], script[]

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: Executing iterative build step

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: TestCase: id[2], script[]

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: Executing iterative build step

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: TestCase: id[2], script[]

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: Executing iterative build step

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: TestCase: id[2], script[]

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: Executing iterative build step

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: TestCase: id[2], script[]

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: Executing iterative build step

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: TestCase: id[2], script[]

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: Executing iterative build step

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: TestCase: id[2], script[]

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: Executing iterative build step

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: TestCase: id[2], script[]

Jul 27, 2012 9:55:19 AM jenkins.plugins.testopia FINE: Executing iterative build step

[tooh]

Additional comments:

All TestCases should be retrieved that :

• match with the TestCaseRun ID as configured in the plugin
• where the Automated field = YES

We have to decide if automated =yes and the script field is empty how to handle this. (Is this FATAL , do we skip or give a warning )

I see in the log every time TestCase: id[2].

I selected TestCaseRun 3

As far as I can see in my testset there are no testcases in TestCaseRun 3.

The logged case

FINE: TestCase: id[2], script[./runjython.sh t/testFinder.py]

is in my testset in TestPlan 3 and Testcase 16

Does this help ?

[kinow]

Hmmm, it does help, thanks Peter. I'll take some screen shots of my environment, and will post my log output here as well, so we can compare our set ups.

[kinow]

Hi Peter, fixed the issue with the case ID, I used using case_status_id instead of case_id.

Now this is my log output:

Aug 4, 2012 12:21:22 AM jenkins.plugins.testopia
FINE: TestCase: id[2], script[blabla]
Aug 4, 2012 12:21:22 AM jenkins.plugins.testopia
FINE: Executing iterative build step
Aug 4, 2012 12:21:22 AM jenkins.plugins.testopia
FINE: TestCase: id[1], script[sample.Test]
Aug 4, 2012 12:21:22 AM jenkins.plugins.testopia
FINE: Executing iterative build step

Could you confirm if you still have the NullPointerException?

Thanks! -B

[tooh]

After this fix my logfile is now :

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: Executing iterative build step

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: TestCase: id[17], script[]

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: Executing iterative build step

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: TestCase: id[16], script[./runjython.sh t/testFinder.py]

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: Executing iterative build step

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: TestCase: id[14], script[]

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: Executing iterative build step

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: TestCase: id[13], script[]

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: Executing iterative build step

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: TestCase: id[12], script[]

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: Executing iterative build step

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: TestCase: id[11], script[]

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: Executing iterative build step

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: TestCase: id[10], script[]

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: Executing iterative build step

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: TestCase: id[9], script[]

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: Executing iterative build step

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: TestCase: id[8], script[]

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: Executing iterative build step

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: TestCase: id[7], script[]

Aug 4, 2012 8:29:23 AM jenkins.plugins.testopia
FINE: Executing iterative build step

My job console output is now :

Started by user anonymous
[EnvInject] - Loading node environment variables.
Building on master in workspace /Users/peter/.jenkins/workspace/TestThemAll - Plugin
Connecting to Testopia to retrieve automated test cases.
Executing single build steps
Executing iterative build steps
[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson1821895728931112720.sh
+ echo 7
7
+ echo

[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson8233852418074350780.sh
+ echo 8
8
+ echo

[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson4615182027869953453.sh
+ echo 9
9
+ echo

[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson3337162221160053045.sh
+ echo 10
10
+ echo

[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson61138034565325349.sh
+ echo 11
11
+ echo

[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson8806387635141276759.sh
+ echo 12
12
+ echo

[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson4764615586354192870.sh
+ echo 13
13
+ echo

[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson2349896110761942129.sh
+ echo 14
14
+ echo

[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson3092169576030015924.sh
+ echo 16
16
+ echo ./runjython.sh t/testFinder.py
./runjython.sh t/testFinder.py
[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson4162374954066409457.sh
+ echo 17
17
+ echo

FATAL: null
java.lang.NullPointerException
    at jenkins.plugins.testthemall.TestopiaBuilder.executeIterativeBuildSteps(TestopiaBuilder.java:251)
    at jenkins.plugins.testthemall.TestopiaBuilder.perform(TestopiaBuilder.java:193)
    at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
    at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)
    at hudson.model.Build$BuildExecution.build(Build.java:199)
    at hudson.model.Build$BuildExecution.doRun(Build.java:160)
    at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
    at hudson.model.Run.execute(Run.java:1488)
    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
    at hudson.model.ResourceController.execute(ResourceController.java:88)
    at hudson.model.Executor.run(Executor.java:236)

[kinow]

Just added an extra verification logic to prevent null test cases... could you give it a try again, please Peter?

Thanks

[tooh]

Yes, no FAILURE anymore.

After fix:

RESULT:

Job succeeds. But I still think I get the wrong results.
Are you sure your query tries to:

• get Testcases that have the flag "Automated" and are part of the selected testrun ?

JOB LOG

Started by user anonymous
[EnvInject] - Loading node environment variables.
Building on master in workspace /Users/peter/.jenkins/workspace/TestThemAll - Plugin
Connecting to Testopia to retrieve automated test cases.
Executing single build steps
Executing iterative build steps
[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson3567464713498332144.sh
+ echo 7
7
+ echo

[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson5120871671176331681.sh
+ echo 8
8
+ echo

[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson4192238859896283262.sh
+ echo 9
9
+ echo

[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson948860812865198704.sh
+ echo 10
10
+ echo

[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson4221184444841962069.sh
+ echo 11
11
+ echo

[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson9068402385977738216.sh
+ echo 12
12
+ echo

[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson4377746723705735664.sh
+ echo 13
13
+ echo

[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson1654029514403613957.sh
+ echo 14
14
+ echo

[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson6909632941191591958.sh
+ echo 16
16
+ echo ./runjython.sh t/testFinder.py
./runjython.sh t/testFinder.py
[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson2718314580812051724.sh
+ echo 17
17
+ echo

Finished: SUCCESS

[kinow]

Yay, we are going in the right direction :-)

Part of the test run, yup... automated... hmmmm, let me check.

[kinow]

There you go https://github.com/kinow/testthemall/commit/da71c377b879d7529e49f0d1bd8c64d95e0acf57

[tooh]

Now I get this:

Started by user anonymous
[EnvInject] - Loading node environment variables.
Building on master in workspace /Users/peter/.jenkins/workspace/TestThemAll - Plugin
Connecting to Testopia to retrieve automated test cases.
FATAL: null
java.lang.NullPointerException
    at jenkins.plugins.testthemall.TestopiaBuilder.filter(TestopiaBuilder.java:212)
    at jenkins.plugins.testthemall.TestopiaBuilder.perform(TestopiaBuilder.java:192)
    at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
    at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)
    at hudson.model.Build$BuildExecution.build(Build.java:199)
    at hudson.model.Build$BuildExecution.doRun(Build.java:160)
    at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
    at hudson.model.Run.execute(Run.java:1488)
    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
    at hudson.model.ResourceController.execute(ResourceController.java:88)
    at hudson.model.Executor.run(Executor.java:236)

[kinow]

Ops hehe, forgot we have a null test case :-) this should do the trick.

[tooh]

Results after fix:

SUCCESS

Only get one case , have to check if this is OK.

Started by user anonymous
[EnvInject] - Loading node environment variables.
Building on master in workspace /Users/peter/.jenkins/workspace/TestThemAll - Plugin
Connecting to Testopia to retrieve automated test cases.
Executing single build steps
Executing iterative build steps
[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson6395487629403975126.sh
+ echo 16
16
+ echo ./runjython.sh t/testFinder.py
./runjython.sh t/testFinder.py
Finished: SUCCESS

[kinow]

Ha! Farewell null testcase! Hope it is the expected result :-)

[tooh]

Test performed

• Created new testplan (ID = 5)
• created new testcase (ID = 54)
• created new testrun (ID = 6)
• associated TC 54 to TCR 6

Build my Jenkins testjob with the plugin:

Got following results:

Started by user anonymous
[EnvInject] - Loading node environment variables.
Building on master in workspace /Users/peter/.jenkins/workspace/TestThemAll - Plugin
Connecting to Testopia to retrieve automated test cases.
Executing single build steps
Executing iterative build steps
[TestThemAll - Plugin] $ /bin/sh -xe /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/hudson225505364836654632.sh
+ echo 54
54
+ echo test.sh
test.sh
Finished: SUCCESS