Set timeout to kill hanged Jenkins run-testplan steps

[kasunbg]

Description:

Sometimes, the builds get stuck during the run-testplan step. See the below log in [1].

The build was stuck at 19:35:59,112. It has not finished even by 23:56:58,308.

[2018-09-01 19:35:58,442]  INFO {org.wso2.carbon.core.services.util.CarbonAuthenticationUtil} -  'admin@carbon.super [-1234]' logged in at [2018-09-01 19:35:58,442+0000]
[19:35:58,471] INFO  [org.wso2.carbon.integration.common.admin.client.AuthenticatorClient] - Login Successful
2018-09-01 19:35:58,536]  INFO {org.wso2.carbon.identity.oauth.config.OAuthServerConfiguration} -  The default OAuth token issuer will be used. No custom token generator is set.
2018-09-01 19:35:58,536]  INFO {org.wso2.carbon.identity.oauth.config.OAuthServerConfiguration} -  The default Identity OAuth token issuer will be used. No custom token generator is set.
[19:35:59,112] INFO  [org.wso2.carbon.automation.engine.testlisteners.TestManagerListener] - =================== Running the test method org.wso2.identity.integration.test.auth.IdentifierFirstLoginTestCase.testIdentifierFirstAuthentication ===================

2018-09-01 23:56:58,308]  INFO {org.wso2.carbon.identity.application.authentication.framework.store.SessionCleanUpService} -  Session Data cleanup task is running successfully for removing expired Data
2018-09-02 00:17:34,890]  INFO {org.wso2.carbon.identity.application.authentication.framework.store.SessionCleanUpService} -  Session Data cleanup task is running successfully for removing expired Data

So, we need to detect and kill any hanging builds. Because of this, the build failure mail is not sent even if one run-testplan got stuck.

[1] https://testgrid-live.private.wso2.com/admin/blue/organizations/jenkins/wso2is-intg/detail/wso2is-intg/30/pipeline/128

[pasindujw]

@kasunbg as a quick solution, how about setting a time-out to the run-testplan?

[wso2-jenkins-bot]

+1. We actually do need a time-out mechanism to detect and kill faulty test-plans. For example, some test scripts may run infinitely due to a bug. So, we need this.

We do need to identify the issue at testgrid level as well. That can come as a separate iteration.

On Sun, Mar 3, 2019 at 3:26 PM Pasindu Jayaweera notifications@github.com wrote:

@kasunbg https://github.com/kasunbg as a quick solution, how about setting a time-out to the run-testplan?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/wso2/testgrid/issues/977#issuecomment-469006771, or mute the thread https://github.com/notifications/unsubscribe-auth/AGb4YCrsad40-c8VRKJola7Nd_uMOC1qks5vS5xdgaJpZM4WWiPB .

-- Kasun Gajasinghe | Technical Lead | WSO2 Inc. (w) +94 11 214 5345 | (e) kasung AT spamfree wso2.com GET INTEGRATION AGILE Integration Agility for Digitally Driven Business