search

eclipse-archived / codewind

The official repository of the Eclipse Codewind project
https://codewind.dev
Eclipse Public License 2.0
113 stars 45 forks source link

[Iterative Dev] Local Functional Test Failing #689

Open maysunfaisal opened 4 years ago

maysunfaisal commented 4 years ago

Codewind version: OS:

Che version: IDE extension version: IDE version: Kubernetes cluster:

Description: Looks like the functional tests in local are failing because Liberty projects cannot build successfully

Steps to reproduce:

Workaround:

localmac-functional-oct15.log

ssh24 commented 4 years ago

Related to https://github.com/eclipse/codewind/issues/693

maysunfaisal commented 4 years ago

Also related to https://github.com/eclipse/codewind/issues/690

maysunfaisal commented 4 years ago 
[18/10/19 03:57:55 codewindtestliberty] [TRACE]  ---> 3ee890be7823

[18/10/19 03:57:55 codewindtestliberty] [TRACE] Successfully built 3ee890be7823

[18/10/19 03:57:56 codewindtestliberty] [TRACE] Successfully tagged cw-codewindtestliberty-liberty1571364537742:latest

[18/10/19 03:57:56 codewindtestliberty] [TRACE] Successfully built cw-codewindtestliberty-liberty1571364537742

[18/10/19 03:57:56 codewindtestliberty] [TRACE] Container image has been built successfully
Updating build state for project codewindtestliberty(liberty1571364537742) to: inProgress

[18/10/19 03:57:56 codewindtestliberty] [ERROR] runScript : 191 | Update build state request failed for project: codewindtestliberty(liberty1571364537742), with code: 404, and message: Not Found

[18/10/19 03:57:56 codewindtestliberty] [TRACE] Container built
* Starting container. 

[18/10/19 03:57:56 codewindtestliberty] [TRACE] Start the container

[18/10/19 03:57:56 codewindtestliberty] [TRACE] Run command to start the container: /file-watcher/idc/artifacts/run_docker.sh cw-codewindtestliberty-liberty1571364537742 cw-codewindtestliberty-liberty1571364537742 "-p 127.0.0.1::7777 -P" /codewind-workspace/.idc/docker/docker-c6f926dbd9ee2b9006ad19e42c84133e14bfafec /codewind-workspace "codewindtestliberty-liberty1571364537742" false

[18/10/19 03:57:56 codewindtestliberty] [TRACE] Running codewind app container for /codewind-workspace/codewindtestliberty using container name cw-codewindtestliberty-liberty1571364537742

[18/10/19 03:57:56 codewindtestliberty] [TRACE] Application path used for volume mounting is: /codewind-workspace/codewindtestliberty
Log path used for volume mounting is: /codewind-workspace/.logs/codewindtestliberty-liberty1571364537742

[18/10/19 03:57:56 codewindtestliberty] [TRACE] 9864a79b3dad131c76abb74c82d9e864d1c85504f77efdd8f6a4ede61d108116

[18/10/19 03:57:57 codewindtestliberty] [TRACE] Stored application container for codewindtestliberty: 9864a79b3dad131c76abb74c82d9e864d1c85504f77efdd8f6a4ede61d108116

[18/10/19 03:57:57 codewindtestliberty] [TRACE] Log file name for build: cw-liberty1571364537742-c6f926dbd9ee2b9006ad19e42c84133e14bfafec

[18/10/19 03:57:57 codewindtestliberty] [TRACE] Compiling application for project codewindtestliberty

[18/10/19 03:57:57 codewindtestliberty] [TRACE] Updating build state for project codewindtestliberty(liberty1571364537742) to: inProgress
[18/10/19 03:57:57 codewindtestliberty] [TRACE] 

[18/10/19 03:57:57 codewindtestliberty] [ERROR] runScript : 191 | Update build state request failed for project: codewindtestliberty(liberty1571364537742), with code: 404, and message: Not Found
[18/10/19 03:57:57 codewindtestliberty] [ERROR] runScript : 191 | 

[18/10/19 03:57:57 codewindtestliberty] [TRACE] Triggering log file event for: maven build log

[18/10/19 03:57:57 codewindtestliberty] [ERROR] runScript : 191 | Update app state request failed for project: codewindtestliberty(liberty1571364537742), with code: 404, and message: Not Found

[18/10/19 03:57:57 codewindtestliberty] [ERROR] runScript : 191 | Application build failed for project codewindtestliberty. Please check the log in the build log view for more details
[18/10/19 03:57:57 codewindtestliberty] [ERROR] runScript : 191 | 

[18/10/19 03:57:57 codewindtestliberty] [TRACE] Updating build state for project codewindtestliberty(liberty1571364537742) to: failed

[18/10/19 03:57:57 codewindtestliberty] [ERROR] runScript : 191 | Update build state request failed for project: codewindtestliberty(liberty1571364537742), with code: 404, and message: Not Found

[18/10/19 03:57:57 codewindtestliberty] [ERROR] runScript : 191 | Application build failed.

[18/10/19 03:57:57 codewindtestliberty] [TRACE] idc build failed for /codewind-workspace/codewindtestliberty with exit code 6 Fri Oct 18 03:57:57 UTC 2019

[18/10/19 03:57:57 codewindtestliberty] [ERROR] runScript : 191 | | The command /file-watcher/scripts/liberty-container.sh /codewind-workspace/codewindtestliberty,/codewind-workspace,liberty1571364537742,rebuild,cw-codewindtestliberty-liberty1571364537742,true,cw-liberty1571364537742-c6f926dbd9ee2b9006ad19e42c84133e14bfafec,debug,7777,NONE,codewindtestliberty-liberty1571364537742 failed with exit code 3
[18/10/19 03:57:57 codewindtestliberty] [ERROR] Project rebuild failed with message: undefined
[18/10/19 03:57:57 codewindtestliberty] [INFO] Emitting event 
 message: projectRestartResult
 data: {
  "operationId": "93e409600d1074b018a44d91838bcc38",
  "projectID": "liberty1571364537742",
  "status": "failed"
}
[18/10/19 03:58:01 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 03:58:03 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
[18/10/19 03:58:11 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 03:58:13 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
[18/10/19 03:58:21 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 03:58:23 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
[18/10/19 03:58:31 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 03:58:33 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
[18/10/19 03:58:41 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 03:58:43 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
[18/10/19 03:58:51 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 03:58:53 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
[18/10/19 03:59:01 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 03:59:03 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
[18/10/19 03:59:11 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 03:59:13 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
[18/10/19 03:59:21 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 03:59:23 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
[18/10/19 03:59:31 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 03:59:33 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
[18/10/19 03:59:41 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 03:59:43 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
[18/10/19 03:59:51 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 03:59:53 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
[18/10/19 04:00:01 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 04:00:03 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
[18/10/19 04:00:11 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 04:00:13 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
[18/10/19 04:00:21 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 04:00:23 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
[18/10/19 04:00:31 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 04:00:33 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
2) "after all" hook: after hook: undefined settings test for "set project action to restart with mode debug"
[18/10/19 04:21:41 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 04:21:43 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
      3) "after all" hook: set app status to running for "set project action with undefined action type"

[18/10/19 04:22:14 codewindtestliberty] [TRACE] idc build finished for /codewind-workspace/codewindtestliberty Fri Oct 18 04:22:14 UTC 2019

[18/10/19 04:22:14 codewindtestliberty] [ERROR] [ File Name: /file-watcher/server/src/projects/projectUtil.ts | Function Name: processManager.spawnDetached | Line Number: 45] Error code: 3 - The container failed to start for application /codewind-workspace/codewindtestliberty
[18/10/19 04:22:14 Turbine] [TRACE] Key translated. Returning message: Creating image for build failed
[18/10/19 04:22:14 codewindtestliberty] [INFO] Validating project using operationId: 39e3f3204bb2ec286191eb14c5885f20
[18/10/19 04:22:14 Turbine] [TRACE] {"projectType":"liberty","location":"/codewind-workspace/codewindtestliberty","projectID":"liberty1571364537742"}
[18/10/19 04:22:14 codewindtestliberty] [INFO] Checking for required files
[18/10/19 04:22:14 codewindtestliberty] [INFO] Checking for required file /Dockerfile | /Dockerfile-lang
[18/10/19 04:22:14 codewindtestliberty] [INFO] Checking for required file /Dockerfile-build
[18/10/19 04:22:14 codewindtestliberty] [TRACE] idc build failed for /codewind-workspace/codewindtestliberty with exit code 6 Fri Oct 18 04:22:14 UTC 2019

[18/10/19 04:22:14 codewindtestliberty] [INFO] Checking for required file /pom.xml
[18/10/19 04:22:14 codewindtestliberty] [ERROR] executeBuildScript : 45 | | The command /file-watcher/scripts/liberty-container.sh /codewind-workspace/codewindtestliberty,/codewind-workspace,liberty1571364537742,update,cw-codewindtestliberty-liberty1571364537742,true,cw-liberty1571364537742-c6f926dbd9ee2b9006ad19e42c84133e14bfafec,run,7777,NONE,codewindtestliberty-liberty1571364537742,, failed with exit code 3
[18/10/19 04:22:14 codewindtestliberty] [INFO] Build state changed for project: liberty1571364537742 from: success, to: failed, with message: buildscripts.buildFail
[18/10/19 04:22:14 codewindtestliberty] [INFO] Emitting event 
 message: projectStatusChanged
 data: {
  "projectID": "liberty1571364537742",
  "buildStatus": "failed",
  "detailedBuildStatus": "Creating image for build failed",
  "lastbuild": 1571372534301
}
[18/10/19 04:22:14 codewindtestliberty] [INFO] Kicking off validation for project: liberty1571364537742
[18/10/19 04:22:14 codewindtestliberty] [INFO] Checking for required file /src/main/liberty/config/server.xml
[18/10/19 04:22:14 codewindtestliberty] [INFO] Sending validation result
[18/10/19 04:22:14 codewindtestliberty] [INFO] Emitted event: projectChanged
{"operationId":"ace87636a3c88cfe7c459951b0e5e1ac","projectID":"liberty1571364537742","ignoredPaths":["/.project","/Dockerfile-tools","/target","/mc-target","/cli-config.yml","/README.md","/Jenkinsfile","/.cfignore","/load-test*","*/node_modules*","*/.git/*","*/.DS_Store","*/*.swp","*/*.swx","*/4913","*/.dockerignore","*/.gitignore","*/*~","/.settings","/localm2cache.zip","/libertyrepocache.zip","/chart"],"isHttps":false,"status":"failed","error":"The container failed to start for application /codewind-workspace/codewindtestliberty"}
[18/10/19 04:22:14 codewindtestliberty] [INFO] Emitting event 
 message: projectValidated
 data: {
  "operationId": "39e3f3204bb2ec286191eb14c5885f20",
  "projectType": "liberty",
  "location": "/codewind-workspace/codewindtestliberty",
  "projectID": "liberty1571364537742",
  "status": "success",
  "results": [],
  "operation": {
    "operationId": "39e3f3204bb2ec286191eb14c5885f20",
    "type": "validate",
    "projectInfo": {
      "projectType": "liberty",
      "location": "/codewind-workspace/codewindtestliberty",
      "projectID": "liberty1571364537742"
    }
  }
}
[18/10/19 04:22:14 codewindtestliberty] [INFO] Emitting event 
 message: projectChanged
 data: {
  "operationId": "ace87636a3c88cfe7c459951b0e5e1ac",
  "projectID": "liberty1571364537742",
  "ignoredPaths": [
    "/.project",
    "/Dockerfile-tools",
    "/target",
    "/mc-target",
    "/cli-config.yml",
    "/README.md",
    "/Jenkinsfile",
    "/.cfignore",
    "/load-test*",
    "*/node_modules*",
    "*/.git/*",
    "*/.DS_Store",
    "*/*.swp",
    "*/*.swx",
    "*/4913",
    "*/.dockerignore",
    "*/.gitignore",
    "*/*~",
    "/.settings",
    "/localm2cache.zip",
    "/libertyrepocache.zip",
    "/chart"
  ],
  "isHttps": false,
  "status": "failed",
  "error": "The container failed to start for application /codewind-workspace/codewindtestliberty"
}
[18/10/19 04:22:21 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 04:22:53 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
[18/10/19 04:23:01 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 04:23:03 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
[18/10/19 04:23:11 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
 1) modify file: Dockerfile
[18/10/19 04:42:13 codewindtestliberty] [ERROR] executeBuildScript : 45 | Application build failed.

[18/10/19 04:42:13 codewindtestliberty] [TRACE] idc build finished for /codewind-workspace/codewindtestliberty Fri Oct 18 04:42:13 UTC 2019

[18/10/19 04:42:13 codewindtestliberty] [ERROR] [ File Name: /file-watcher/server/src/projects/projectUtil.ts | Function Name: processManager.spawnDetached | Line Number: 45] Error code: 3 - The container failed to start for application /codewind-workspace/codewindtestliberty
[18/10/19 04:42:13 codewindtestliberty] [INFO] Emitted event: projectChanged
{"operationId":"7f941d9a8898429582ef542f6eefcc0e","projectID":"liberty1571364537742","ignoredPaths":["/.project","/Dockerfile-tools","/target","/mc-target","/cli-config.yml","/README.md","/Jenkinsfile","/.cfignore","/load-test*","*/node_modules*","*/.git/*","*/.DS_Store","*/*.swp","*/*.swx","*/4913","*/.dockerignore","*/.gitignore","*/*~","/.settings","/localm2cache.zip","/libertyrepocache.zip","/chart"],"isHttps":false,"status":"failed","error":"The container failed to start for application /codewind-workspace/codewindtestliberty"}
[18/10/19 04:42:13 codewindtestliberty] [TRACE] idc build failed for /codewind-workspace/codewindtestliberty with exit code 6 Fri Oct 18 04:42:13 UTC 2019

[18/10/19 04:42:13 codewindtestliberty] [ERROR] executeBuildScript : 45 | | The command /file-watcher/scripts/liberty-container.sh /codewind-workspace/codewindtestliberty,/codewind-workspace,liberty1571364537742,update,cw-codewindtestliberty-liberty1571364537742,true,cw-liberty1571364537742-c6f926dbd9ee2b9006ad19e42c84133e14bfafec,run,7777,NONE,codewindtestliberty-liberty1571364537742,, failed with exit code 3
[18/10/19 04:42:13 codewindtestliberty] [INFO] Emitting event 
 message: projectChanged
 data: {
  "operationId": "7f941d9a8898429582ef542f6eefcc0e",
  "projectID": "liberty1571364537742",
  "ignoredPaths": [
    "/.project",
    "/Dockerfile-tools",
    "/target",
    "/mc-target",
    "/cli-config.yml",
    "/README.md",
    "/Jenkinsfile",
    "/.cfignore",
    "/load-test*",
    "*/node_modules*",
    "*/.git/*",
    "*/.DS_Store",
    "*/*.swp",
    "*/*.swx",
    "*/4913",
    "*/.dockerignore",
    "*/.gitignore",
    "*/*~",
    "/.settings",
    "/localm2cache.zip",
    "/libertyrepocache.zip",
    "/chart"
  ],
  "isHttps": false,
  "status": "failed",
  "error": "The container failed to start for application /codewind-workspace/codewindtestliberty"
}
[18/10/19 04:42:21 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 04:42:49 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
[18/10/19 04:42:51 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error
[18/10/19 04:42:53 codewindtestliberty] [INFO] pingApplications: Application state error message: Error: connect EHOSTUNREACH 172.19.0.4:9080
[18/10/19 04:43:01 codewindtestliberty] [INFO] pingApplications: Application state error message: Http request is failed due to timeout error

 2) modify file: src/main/java/application/rest/v1/Example.java

other failures: 1) create file: file1 1) get project logs with valid id

And test seems to time out after this test

  ✓ checking for new build log file: 5004ms
[18/10/19 05:21:54 codewindtestliberty] [INFO] buildlog path:
 {"origin":"workspace","files":["/codewind-workspace/.logs/codewindtestliberty-liberty1571364537742/docker.build.log"]}
[18/10/19 05:21:54 codewindtestliberty] [INFO] applog path:
{"origin":"workspace","files":[]}
[18/10/19 05:21:54 codewindtestliberty] [INFO] buildlog path:
 {"origin":"workspace","files":["/codewind-workspace/.logs/codewindtestliberty-liberty1571364537742/docker.build.log"]}
[18/10/19 05:21:54 codewindtestliberty] [INFO] applog path:
{"origin":"workspace","files":[]}
[18/10/19 05:21:54 codewindtestliberty] [INFO] buildlog path:
 {"origin":"workspace","files":["/codewind-workspace/.logs/codewindtestliberty-liberty1571364537742/docker.build.log"]}
[18/10/19 05:21:54 codewindtestliberty] [INFO] applog path:

with 

 {"origin":"workspace","files":["/codewind-workspace/.logs/codewindtestliberty-liberty1571364537742/docker.build.log"]}
[18/10/19 05:21:54 codewindtestliberty] [INFO] applog path:

being spammed until eternity

yangcao77 commented 4 years ago

Unrelated to this issue, but I do find the log being spammed a lot with the applog path a buildlog path. Can we remove the logging message?

ssh24 commented 4 years ago

The logging happens when it can't find the required files. And usually it cannot find the required log files if it failed on a step. But yeah, we can remove the logging message.