Need to log an error if scenario run script is not found.

wso2 / testgrid

TestGrid provides the enterprise customers confidence on the products and updates WSO2 ship.

Apache License 2.0

55 stars 66 forks source link

Description: Currently, the run-testplan fails silently if the run-scenario.sh script is not found. It is not clear how to solve the issue.

See the below log I got for test-scripts/local-is-deployment/ sample.

[15:56:41,763] TESTGRID_HOME environment variable not set. Defaulting to ~/.testgrid.
[15:56:41,791] One or more database properties {DB_URL, DB_USER, DB_USER_PASS} are not set in config.properties. Using default properties in persistence.xml
[15:56:43,396] TestGrid Home    : /home/kasun/.testgrid
[15:56:43,717] Executing provisioning scripts...
[15:56:43,738] This is a Dummy Infrastructure provisioning scripts, Your local machine will be used. Hence, infra-provision is empty.
[15:56:43,739] Start executing the deployment script
[15:56:43,739] /home/kasun/git-repos/testgrid
[15:56:43,740] Performing the Deployment 01-local-is-deployment
[15:56:43,741] Performing the Deployment deploy-SDGODI
[15:56:43,746] + echo 'This is a Dummy Infrastructure provisioning scripts, Your local machine will be used. Hence, infra-provision is empty.'
[15:56:43,747] + echo 'Start executing the deployment script'
[15:56:43,747] ++ pwd
[15:56:43,748] + CURRENT_PATH=/home/kasun/git-repos/testgrid
[15:56:43,748] + echo /home/kasun/git-repos/testgrid
[15:56:43,756] This is a Dummy Deployment creation scripts to deploy a standalone WSO2 Identity server
[15:56:43,755] cat: /home/kasun/.testgrid/wso2is/01-local-is-deployment/dc800518-13d9-33c6-b1da-56fca034cb5f/1/k8s.properties: No such file or directory
[15:56:43,756] Here the test infrastructure is your local machine.
[15:56:43,756] unzip the IS distribution
[15:56:43,759] + echo 'This is a Dummy Deployment creation scripts to deploy a standalone WSO2 Identity server'
[15:56:43,759] + echo 'Here the test infrastructure is your local machine.'
[15:56:43,759] + DEPLOYMENT_EP_FILE_NAME=deployment_eps
[15:56:43,759] + echo 'unzip the IS distribution'
[15:56:43,759] + unzip -q -o wso2is-5.4.0.zip
[15:56:46,227] + echo -n 'Starting the IS server'
[15:56:46,228] + bash ./wso2is-5.4.0/bin/wso2server.sh start
[15:56:46,260] Starting the IS serverProcess is already running
[15:56:46,260] + x=0
[15:56:46,260] Wait for the server start up
[15:56:46,260] + retry_count=60
[15:56:46,261] + echo 'Wait for the server start up'
[15:56:46,261] + nc -z localhost 9763
[15:56:46,262] Successfully started the server!!
[15:56:46,262] + echo 'Successfully started the server!!'
[15:56:46,262] Generating the deployment_ep file with the endpoints
[15:56:46,262] + echo 'Generating the deployment_ep file with the endpoints'
[15:56:46,262] Deployment has finished, test execution will commence next
[15:56:46,262] + echo https://localhost:9443/carbon
[15:56:46,262] + echo 'Deployment has finished, test execution will commence next'
[15:56:46,265] The deployment.json file was not found at: /home/kasun/.testgrid/wso2is/01-local-is-deployment/dc800518-13d9-33c6-b1da-56fca034cb5f/1/deployment.json. This is where the deployment creation outputs are stored. Continuing without the deployment outputs.
[15:56:46,272] Executing Tests for Solution Pattern : Test scenario 01
[15:56:46,279] Couldn't find any tests for the scenario TestScenario{id='3a35471b-8491-4006-834c-594239544df9', name='Test scenario 01', status='RUNNING', createdTimestamp='2018-05-08 15:27:44.0', modifiedTimestamp='2018-05-08 15:56:29.0', testPlan='TestPlan{id='c92ef443-44a8-4305-8474-a21cbd320966', status='RUNNING', testRunNumber='1', createdTimestamp='', modifiedTimestamp='', deploymentPattern='DeploymentPattern{id='98d263e4-d695-497e-875e-c78a7a9ae6a6', name='01-local-is-deployment', createdTimestamp='2018-05-08 15:27:44.0', modifiedTimestamp='2018-05-08 15:27:44.0', product='Product{id='3114f31f-b47f-4975-805e-5d9d95b3e2bc', name='wso2is', createdTimestamp='2018-05-08 15:27:44.0', modifiedTimestamp='2018-05-08 15:27:44.0'}'}', infrastructureRepository='/home/kasun/git-repos/testgrid/test-scripts/local-is-deployment/Infrastructure', deploymentRepository='/home/kasun/git-repos/testgrid/test-scripts/local-is-deployment/Deployment', scenarioTestsRepository='/home/kasun/git-repos/testgrid/test-scripts/local-is-deployment/Solutions'}'} At location /home/kasun/git-repos/testgrid/test-scripts/local-is-deployment/Solutions/solution01
[15:56:49,401] Destroying test environment...
[15:56:49,410] + echo 'This is a Dummy Infrastructure destroy scripts!!'
[15:56:49,410] This is a Dummy Infrastructure destroy scripts!!
[15:56:49,412] Stopping and cleaning the IS server
[15:56:49,417] + echo 'Stopping and cleaning the IS server'
[15:56:49,417] + bash ./wso2is-5.4.0/bin/wso2server.sh stop
[15:56:49,420] bash: ./wso2is-5.4.0/bin/wso2server.sh: No such file or directory
[15:56:49,421] + rm -rf wso2is-5.4.0
[15:56:49,428] There are test failures...
[15:56:49,428] Failed tests:
[15:56:53,244] 
[15:56:53,244] Tests run: 0, Failures/Errors: 0
[15:56:53,245] 
[15:56:53,245] ------------------------------------------------------------------------
[15:56:53,251] Test Plan Summary for {"DBEngineVersion":"5.6.35"}
[15:56:53,251] Test scenario 01 ................................... ERROR
[15:56:53,252] ------------------------------------------------------------------------
[15:56:53,253] TEST RUN ERROR
[15:56:53,253] ------------------------------------------------------------------------
[15:56:53,254] Total Time: 5 seconds
[15:56:53,254] Finished at: Tue May 08 15:56:53 IST 2018
[15:56:53,255] ------------------------------------------------------------------------

Affected Product Version: 0.9.0-m25

OS, DB, other environment details and versions:

Steps to reproduce: Execute run-testplan for test-scripts/local-is-deployment

Related Issues:

[pool-2-thread-1] INFO org.wso2.testgrid.common.ShellExecutor - CARBON cannot execute java [pool-2-thread-2] ERROR org.wso2.testgrid.common.ShellExecutor - + x=0 [pool-2-thread-1] INFO org.wso2.testgrid.common.ShellExecutor - Wait for the server start up [pool-2-thread-2] ERROR org.wso2.testgrid.common.ShellExecutor - + retry_count=60 [pool-2-thread-2] ERROR org.wso2.testgrid.common.ShellExecutor - + echo 'Wait for the server start up' [pool-2-thread-2] ERROR org.wso2.testgrid.common.ShellExecutor - + nc -z localhost 9763 [pool-2-thread-2] ERROR org.wso2.testgrid.common.ShellExecutor - + echo 'Successfully started the server!!' [pool-2-thread-2] ERROR org.wso2.testgrid.common.ShellExecutor - + echo 'Generating the deployment_ep file with the endpoints' [pool-2-thread-2] ERROR org.wso2.testgrid.common.ShellExecutor - + echo https://localhost:9443/carbon [pool-2-thread-1] INFO org.wso2.testgrid.common.ShellExecutor - Successfully started the server!! [pool-2-thread-1] INFO org.wso2.testgrid.common.ShellExecutor - Generating the deployment_ep file with the endpoints [pool-2-thread-2] ERROR org.wso2.testgrid.common.ShellExecutor - + echo 'Deployment has finished, test execution will commence next' [pool-2-thread-1] INFO org.wso2.testgrid.common.ShellExecutor - Deployment has finished, test execution will commence next [main] WARN org.wso2.testgrid.deployment.DeploymentUtil - The deployment.json file was not found at: /home/pasindu/Developments/qgrid/temp/local-is-deployment/IS/01-local-is-deployment/bdfe164d-492d-3473-a947-ba07c1e9ff61/7/deployment.json. This is where the deployment creation outputs are stored. Continuing without the deployment outputs. [main] INFO org.wso2.testgrid.core.ScenarioExecutor - Executing Tests for Solution Pattern : Test scenario 01 [main] WARN org.wso2.testgrid.core.ScenarioExecutor - Couldn't find any tests for the scenario TestScenario{id='73f18e7a-015c-4aea-bd61-aee2772ffa90', name='Test scenario 01', status='RUNNING', createdTimestamp='2018-05-22 16:37:35.0', modifiedTimestamp='2018-05-23 10:07:06.0', testPlan='TestPlan{id='56a12089-4a9c-4442-9540-02b81a9fff7f', status='RUNNING', testRunNumber='7', createdTimestamp='', modifiedTimestamp='', deploymentPattern='DeploymentPattern{id='af7c62b1-33b7-42f2-bde8-30d1030827ea', name='01-local-is-deployment', createdTimestamp='2018-05-22 10:40:24.0', modifiedTimestamp='2018-05-22 10:40:24.0', product='Product{id='b5e905bc-97b4-4487-8e57-a0512bceba12', name='IS', createdTimestamp='2018-05-22 10:40:24.0', modifiedTimestamp='2018-05-22 10:40:24.0'}'}', infrastructureRepository='/home/pasindu/Developments/qgrid/temp/local-is-deployment/Infrastructure', deploymentRepository='/home/pasindu/Developments/qgrid/temp/local-is-deployment/Deployment', scenarioTestsRepository='/home/pasindu/Developments/qgrid/temp/local-is-deployment/Solutions'}'} At location /home/pasindu/Developments/qgrid/temp/local-is-deployment/Solutions/solution01 [main] INFO org.wso2.testgrid.core.ScenarioExecutor - Executing Tests for Solution Pattern : Test scenario 03 [main] INFO org.wso2.testgrid.core.ScenarioExecutor - Executing Test scenario 03 Test [main] INFO org.wso2.testgrid.common.util.TestGridUtil - Execution result : This is a Dummy Scenario execution scripts Created the tree successfully using jmeter/01-TestIS540.jmx Starting the test @ Wed May 23 10:08:07 IST 2018 (1527050287541) Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445 Tidying up ... @ Wed May 23 10:08:09 IST 2018 (1527050289090) ... end of run after [main] INFO org.wso2.testgrid.core.ScenarioExecutor - --------------------------------------- [main] INFO org.wso2.testgrid.core.ScenarioExecutor - Executing Tests for Solution Pattern : Test scenario 02 [main] INFO org.wso2.testgrid.core.ScenarioExecutor - Executing Test scenario 02 Test [main] INFO org.wso2.testgrid.common.util.TestGridUtil - Execution result : This is a Dummy Scenario execution scripts Created the tree successfully using jmeter/01-TestIS540.jmx Starting the test @ Wed May 23 10:08:10 IST 2018 (1527050290665) Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445 Tidying up ... @ Wed May 23 10:08:12 IST 2018 (1527050292253) ... end of run after [main] INFO org.wso2.testgrid.core.ScenarioExecutor - --------------------------------------- [main] INFO org.wso2.testgrid.infrastructure.providers.ShellScriptProvider - Destroying test environment... [pool-3-thread-1] INFO org.wso2.testgrid.common.ShellExecutor - This is a Dummy Infrastructure destroy scripts!! [pool-3-thread-1] INFO org.wso2.testgrid.common.ShellExecutor - Stopping and cleaning the IS server [pool-3-thread-2] ERROR org.wso2.testgrid.common.ShellExecutor - + echo 'This is a Dummy Infrastructure destroy scripts!!' [pool-3-thread-2] ERROR org.wso2.testgrid.common.ShellExecutor - + echo 'Stopping and cleaning the IS server' [pool-3-thread-2] ERROR org.wso2.testgrid.common.ShellExecutor - + bash ./wso2is-5.4.0/bin/wso2server.sh stop [pool-3-thread-2] ERROR org.wso2.testgrid.common.ShellExecutor - bash: ./wso2is-5.4.0/bin/wso2server.sh: No such file or directory [pool-3-thread-2] ERROR org.wso2.testgrid.common.ShellExecutor - + rm -rf wso2is-5.4.0 Process finished with exit code 0

wso2 / testgrid

Need to log an error if scenario run script is not found. #743