eclipse-che / che

Kubernetes based Cloud Development Environments for Enterprise Teams
http://eclipse.org/che
Eclipse Public License 2.0
6.99k stars 1.19k forks source link

$PROJECT_SOURCE incorrectly set when starterProjects are used in devfile located in git repository #23065

Open AObuchow opened 2 months ago

AObuchow commented 2 months ago

Describe the bug

Creating a workspace from a (repository-hosted) devfile that defines only a starter project will result in $PROJECT_SOURCE getting incorrectly set to an implicit project that points to the devfile's repository location.

Che version

7.89@latest

Steps to reproduce

  1. Create a workspace from a devfile that contains only a starterProject, such as this one. The devfile must be hosted in a git repository.
  2. Create a terminal in CheCode and run echo $PROJECT_SOURCE
  3. The value of $PROJECT_SOURCE is set to the implicit devfile git repository project created by the Dashboard, in this case

Expected behavior

The value of $PROJECT_SOURCE should be the value of the projected selected with the controller.devfile.io/use-starter-project devworkspace attribute. In this case, $PROJECT_SOURCE should be /projects/flask-example/.

Note: The DevWorkspace has the correct controller.devfile.io/use-starter-project value:

$ cat /devworkspace-metadata/original.devworkspace.yaml | grep "use-starter-project"
  controller.devfile.io/use-starter-project: flask-example

Runtime

OpenShift

Screenshots

No response

Installation method

OperatorHub

Environment

Linux

Eclipse Che Logs

No response

Additional context

The root cause of this problem stems from a conflict with the Che Dashboard and DevWorkspace Operator:

This issue is quite low severity, as it only applies when using a devfile hosted in a git repository. If you use the same raw devfile where it's hosted on the devfile registry (https://registry.devfile.io/devfiles/python/3.1.0) the bug does not occur.

AObuchow commented 5 days ago

A customer has encountered a variant of this same bug. In the customer's case, they had a parent devfile that defines a starterProject as well as devfile commands that are intended to run in the starterProject's directory (using $PROJECT_SOURCE). The expected result is that the starterProject is used, but instead, the implicit project (based on the devfiles git repo location) is used. As a result, the starterProject is not cloned & the devfile commands do not work.

I've created the following repository to reproduce this bug.

Here is the child devfile:

schemaVersion: 2.3.0
metadata:
  name: child-devfile-reproducer
parent:
  uri: https://raw.githubusercontent.com/AObuchow/parent-devfile-starterProject-reproducer/refs/heads/main/parent-devfile.yaml

And here is the parent devfile:

schemaVersion: 2.3.0
metadata:
  name: parent-devfile-reproducer
starterProjects:
  - name: flask-example
    git:
      remotes:
        origin: "https://github.com/devfile-samples/python-ex"
components:
  - name: py
    container:
      image: quay.io/devfile/universal-developer-image:ubi8-latest
      args: ['tail', '-f', '/dev/null']
      mountSources: true
      endpoints:
        - name: https-python
          targetPort: 8080
          protocol: http
          secure: true
          attributes:
            discoverable: true
        - exposure: none
          name: debug
          targetPort: 5858
      env:
        - name: DEBUG_PORT
          value: '5858'
commands:
  - id: pip-install-requirements
    exec:
      commandLine: pip install -r requirements.txt
      workingDir: ${PROJECT_SOURCE}
      group:
        kind: build
        isDefault: true
      component: py
  - id: run-app
    exec:
      commandLine: 'python app.py'
      workingDir: ${PROJECT_SOURCE}
      component: py
      group:
        kind: run
        isDefault: true
  - id: debug-py
    exec:
      commandLine: 'pip install debugpy && python -m debugpy --listen 0.0.0.0:${DEBUG_PORT} app.py'
      workingDir: ${PROJECT_SOURCE}
      component: py
      group:
        kind: debug

Here is the resulting devworkspace (from the /devworkspace-metadata/ directory in the workspace container:

attributes:
  controller.devfile.io/devworkspace-config:
    name: devworkspace-config
    namespace: dogfooding
  controller.devfile.io/scc: container-build
  controller.devfile.io/storage-type: ephemeral
  dw.metadata.annotations:
    che.eclipse.org/devfile-source: |
      scm:
        repo: https://github.com/AObuchow/parent-devfile-starterProject-reproducer.git
        revision: main
        fileName: devfile.yaml
      factory:
        params: >-
          storageType=ephemeral&url=https://github.com/AObuchow/parent-devfile-starterProject-reproducer/tree/main
parent:
  uri: https://raw.githubusercontent.com/AObuchow/parent-devfile-starterProject-reproducer/refs/heads/main/parent-devfile.yaml
projects:
- git:
    checkoutFrom:
      revision: main
    remotes:
      origin: https://github.com/AObuchow/parent-devfile-starterProject-reproducer.git
  name: parent-devfile-starterproject-reproducer #This is the implicit project added by the Dashboard

And here's the flattened workspace:

attributes:
  controller.devfile.io/devworkspace-config:
    name: devworkspace-config
    namespace: dogfooding
  controller.devfile.io/scc: container-build
  controller.devfile.io/storage-type: ephemeral
  dw.metadata.annotations:
    che.eclipse.org/devfile-source: |
      scm:
        repo: https://github.com/AObuchow/parent-devfile-starterProject-reproducer.git
        revision: main
        fileName: devfile.yaml
      factory:
        params: >-
          storageType=ephemeral&url=https://github.com/AObuchow/parent-devfile-starterProject-reproducer/tree/main
commands:
- attributes:
    controller.devfile.io/imported-by: parent
  exec:
    commandLine: pip install -r requirements.txt
    component: py
    group:
      isDefault: true
      kind: build
    workingDir: ${PROJECT_SOURCE}
  id: pip-install-requirements
- attributes:
    controller.devfile.io/imported-by: parent
  exec:
    commandLine: python app.py
    component: py
    group:
      isDefault: true
      kind: run
    workingDir: ${PROJECT_SOURCE}
  id: run-app
- attributes:
    controller.devfile.io/imported-by: parent
  exec:
    commandLine: pip install debugpy && python -m debugpy --listen 0.0.0.0:${DEBUG_PORT}
      app.py
    component: py
    group:
      kind: debug
    workingDir: ${PROJECT_SOURCE}
  id: debug-py
- apply:
    component: che-code-injector
  attributes:
    controller.devfile.io/imported-by: editor
  id: init-container-command
- attributes:
    controller.devfile.io/imported-by: editor
  exec:
    commandLine: nohup /checode/entrypoint-volume.sh > /checode/entrypoint-logs.txt
      2>&1 &
    component: py
  id: init-che-code-command
- exec:
    commandLine: |-
      SSH_ENV_PATH=$HOME/ssh-environment \
      && if [ -f /etc/ssh/passphrase ] && command -v ssh-add >/dev/null; \
      then ssh-agent | sed 's/^echo/#echo/' > $SSH_ENV_PATH \
      && chmod 600 $SSH_ENV_PATH && source $SSH_ENV_PATH \
      && ssh-add /etc/ssh/dwo_ssh_key < /etc/ssh/passphrase \
      && if [ -f $HOME/.bashrc ] && [ -w $HOME/.bashrc ]; then echo "source ${SSH_ENV_PATH}" >> $HOME/.bashrc; fi; fi
    component: py
  id: init-ssh-agent
components:
- attributes:
    app.kubernetes.io/component: che-code-runtime
    app.kubernetes.io/part-of: che-code.eclipse.org
    controller.devfile.io/imported-by: parent
    controller.devfile.io/merged-contributions: editor
  container:
    args:
    - tail
    - -f
    - /dev/null
    endpoints:
    - attributes:
        controller.devfile.io/endpoint-url: https://che-dogfooding.apps.che-dev.x6e0.p1.openshiftapps.com/aobuchow/child-devfile-reproducer/3100/
        cookiesAuthEnabled: true
        discoverable: false
        type: main
        urlRewriteSupported: true
      exposure: public
      name: che-code
      protocol: https
      secure: true
      targetPort: 3100
    - attributes:
        controller.devfile.io/endpoint-url: https://aobuchow-child-devfile-reproducer-code-redirect-1.apps.che-dev.x6e0.p1.openshiftapps.com/
        discoverable: false
        urlRewriteSupported: false
      exposure: public
      name: code-redirect-1
      protocol: https
      targetPort: 13131
    - attributes:
        controller.devfile.io/endpoint-url: https://aobuchow-child-devfile-reproducer-code-redirect-2.apps.che-dev.x6e0.p1.openshiftapps.com/
        discoverable: false
        urlRewriteSupported: false
      exposure: public
      name: code-redirect-2
      protocol: https
      targetPort: 13132
    - attributes:
        controller.devfile.io/endpoint-url: https://aobuchow-child-devfile-reproducer-code-redirect-3.apps.che-dev.x6e0.p1.openshiftapps.com/
        discoverable: false
        urlRewriteSupported: false
      exposure: public
      name: code-redirect-3
      protocol: https
      targetPort: 13133
    - attributes:
        controller.devfile.io/endpoint-url: https://aobuchow-child-devfile-reproducer-https-python.apps.che-dev.x6e0.p1.openshiftapps.com/
        discoverable: true
      name: https-python
      protocol: http
      secure: true
      targetPort: 8080
    - exposure: none
      name: debug
      targetPort: 5858
    env:
    - name: CHE_DASHBOARD_URL
      value: https://che-dogfooding.apps.che-dev.x6e0.p1.openshiftapps.com
    - name: CHE_PLUGIN_REGISTRY_URL
      value: ""
    - name: CHE_PLUGIN_REGISTRY_INTERNAL_URL
      value: ""
    - name: CLUSTER_CONSOLE_URL
      value: https://console-openshift-console.apps.che-dev.x6e0.p1.openshiftapps.com
    - name: CLUSTER_CONSOLE_TITLE
      value: OpenShift console
    - name: OPENVSX_REGISTRY_URL
      value: https://open-vsx.org
    - name: DEBUG_PORT
      value: "5858"
    image: quay.io/devfile/universal-developer-image:ubi8-latest
    memoryLimit: 1152Mi
    memoryRequest: 320Mi
    mountSources: true
    sourceMapping: /projects
    volumeMounts:
    - name: checode
      path: /checode
  name: py
- attributes:
    controller.devfile.io/imported-by: editor
  container:
    command:
    - /entrypoint-init-container.sh
    cpuLimit: 500m
    cpuRequest: 30m
    env:
    - name: CHE_DASHBOARD_URL
      value: https://che-dogfooding.apps.che-dev.x6e0.p1.openshiftapps.com
    - name: CHE_PLUGIN_REGISTRY_URL
      value: ""
    - name: CHE_PLUGIN_REGISTRY_INTERNAL_URL
      value: ""
    - name: CLUSTER_CONSOLE_URL
      value: https://console-openshift-console.apps.che-dev.x6e0.p1.openshiftapps.com
    - name: CLUSTER_CONSOLE_TITLE
      value: OpenShift console
    - name: OPENVSX_REGISTRY_URL
      value: https://open-vsx.org
    image: quay.io/che-incubator/che-code:insiders
    memoryLimit: 256Mi
    memoryRequest: 32Mi
    sourceMapping: /projects
    volumeMounts:
    - name: checode
      path: /checode
  name: che-code-injector
- attributes:
    controller.devfile.io/imported-by: editor
  name: checode
  volume: {}
events:
  postStart:
  - init-che-code-command
  - init-ssh-agent
  preStart:
  - init-container-command
projects:
- git:
    checkoutFrom:
      revision: main
    remotes:
      origin: https://github.com/AObuchow/parent-devfile-starterProject-reproducer.git
  name: parent-devfile-starterproject-reproducer
starterProjects:
- attributes: # Missing the `controller.devfile.io/use-starter-project` attribute
    controller.devfile.io/imported-by: parent 
  git:
    remotes:
      origin: https://github.com/devfile-samples/python-ex
  name: flask-example

Note: in this case, the controller.devfile.io/use-starter-project attribute is missing from the devworkspace.

Workaround

As a temporary workaround, you can change starterProjects to projects in the parent devfile. This results in both the implicit project being cloned into the workspace, as well as the flask-example project. $PROJECT_SOURCE will also be set correctly to /projects/flask-example