SVT:[che]:Appsody projects status stuck in "starting" after workspace restart due to user session timeout

[jagraj]

Codewind version: 0.6.0 OS: RHEL

Che version: 7.3.1 IDE extension version: 0.6 IDE version: 7.3.1 Kubernetes cluster:

OS 3.11 cluster

Description: All appsody projects "status" is in "starting" state after workspace restart due to session timeout. I do not see this problem with default codewind projects.

Steps to reproduce:

1. Install Codewind Che from 0.6.0 plugin
2. After creating all appsody and default codewind projects and wait until all projects up and running.
3. After session timeout, the workspace is restarted the entire workspace is being rebuilt.
4. The status for all appsody projects stuck in starting state but not for default codewind templates.

Logs pfe.txt theia.txt

Workaround:

[sghung]

I asked Andrew to take a look at this one.

[makandre]

Where are the appsody logs?

[makandre]

[32m[18/11/19 20:39:11 appsodynodejscw06svt] [INFO] Emitted event: projectCreation
{"operationId":"0b156bcda276ddff33c8363ca4f59627","projectID":"2a822900-0a0c-11ea-86f6-a532b5f1f0f8","isHttps":false,"status":"success","ports":{},"logs":{"build":[],"app":[{"origin":"workspace","files":["/codewind-workspace/.logs/appsodynodejscw06svt-2a822900-0a0c-11ea-86f6-a532b5f1f0f8/appsody.log"]}]},"error":{}}
[18/11/19 20:39:11 appsodynodejscw06svt] [ERROR] Unable to retrieve project deployment or service
[18/11/19 20:39:11 appsodynodejscw06svt] [INFO] Emitting event 
 message: projectCreation
 data: {
  "operationId": "0b156bcda276ddff33c8363ca4f59627",
  "projectID": "2a822900-0a0c-11ea-86f6-a532b5f1f0f8",
  "isHttps": false,
  "status": "success",
  "ports": {},
  "logs": {
    "build": [],
    "app": [
      {
        "origin": "workspace",
        "files": [
          "/codewind-workspace/.logs/appsodynodejscw06svt-2a822900-0a0c-11ea-86f6-a532b5f1f0f8/appsody.log"
        ]
      }
    ]
  },
  "error": {}
}
[18/11/19 20:39:12 swiftcw06svt] [INFO] Pod name was not found while looking up service information for project.
[18/11/19 20:39:48 appsodyspringcw06svt] [INFO] pingInTransitApplications: Application state error message: could not retrieve the port for the project

This part in the log is telling. At least for this particular case, it seems Codewind can't find the application port, and it's because it can't find the appsody deployment/service.

There's a wait we do here https://github.com/eclipse/codewind-appsody-extension/blob/master/scripts/wait-for-container.sh#L20 for the pod to come up. With so many projects/pods/services being created at the same time, it's possible the wait times out before the appsody pod comes up.

[jagraj]

I got into cluster issues and I could n't install che. After rebooting cluster, I am able to install che on my cluster. I will try to recreate and update my results here.

[makandre]

@jagraj looks like we need a new build of PFE to pick up the new change anyways @sghung or @josephca do we have other fixes going into PFE?

[josephca]

@makandre I believe this new build already picked up the new changes. Please let me know otherwise. https://ci.eclipse.org/codewind/view/Codewind/job/Codewind/job/codewind/job/0.6.0/13

[jagraj]

I got the latest images and verified this fix after workspace restart and projects are no longer stuck in "Starting" state. Thanks for the fix.