Add ability to "bootstrap" devworkspaces in project-clone

[amisevsk]

Description

In many cases, it would be useful to be able to "bootstrap" a DevWorkspace from a devfile. For example, a user could create a basic DevWorkspace and configure it so that the project-clone container:

1. Clones a project via Git
2. Checks the cloned project for a devfile.yaml or .devfile.yaml
3. Updates the DevWorkspace's .spec.template to match the detected devfile, causing the workspace to restart.

This would make creating DevWorkspaces in certain contexts a lot easier; if git credentials are configured on a cluster, the project-clone container could clone a private or otherwise inaccessible project and set up a DevWorkspace for it automatically.

This could be controlled via a devfile attribute, e.g. controller.devfile.io/bootstrap-devworkspace: true to trigger this behavior.

Additional context

[l0rd]

Che dashboard, che devfile registry build and che code are all using the same npm library to generate DevWorkspaces. We should avoid to re-create a generator and use that instead.

[amisevsk]

The generator library isn't suitable for use in DWO for a number of reasons, a large one being that we don't want to interface with an npm library within the golang-based project.

Apart from that, the library depends on a lot of features external to DWO to work (e.g. injecting a default component, setting environment variables for a Che install, etc.)

[l0rd]

I agree with you. This is not ideal at all. But duplicating the logic to generate devfiles in 2 different libraries, in 2 different languages, is worst. At minimum we will have to fix bugs twice, at worst we will have 2 inconsistent implementations. Rather than investing in new code we should consider refactoring the existing library so that we extract Che specific features. The library cannot be loaded in go code but should execute it as a command line utility.

[amisevsk]

The issue is that the devfile generation library does 2-3 things when we are only concerned with one of them here -- we already have a DevWorkspace with default editor, env vars, etc. For example, within DWO, we can't change the name of the DevWorkspace, as that would be the same as deleting the current workspace and creating a new one.

Ideally, we instead adapt the library so that generated DevWorkspaces don't modify the underlying devfile, minimizing the amount of actual generating required. At the level DWO operates at, its understanding of devfiles and DevWorkspaces amounts to basically "a devfile represents the .spec.template of a DevWorkspace", which leads to a fairly straightforward conversion process:

1. Get devfile content
2. Copy attributes, components, commands, events, projects from devfile to DevWorkspace's .spec.template
3. Convert devfile metadata to DevWorkspace metadata (basically just the name)
4. Set DevWorkspace-specific fields as appropriate (started, routingClass, etc.)

This makes an implementation of this in project clone fairly simple, without requiring use of an npm library:

1. Clone projects
2. Find devfile (search cloned projects in order looking for a devfile.yaml; ignore starterProjects)
3. Read devfile, replace .spec.template as above

[l0rd]

The issue is that the devfile generation library does 2-3 things when we are only concerned with one of them here -- we already have a DevWorkspace with default editor, env vars, etc. For example, within DWO, we can't change the name of the DevWorkspace, as that would be the same as deleting the current workspace and creating a new one.

To me this is the same use case as Che Code "Restart from local Devfile" (and the dwgenerator is used). Please let's not re-invent the wheel.

[amisevsk]

I've opened PR https://github.com/devfile/devworkspace-operator/pull/1193, which resolves the issue.

As to the question of reinventing the wheel, no logic from the npm generator library is used or needed (short of setting workspace.spec.template = <devfile>, since we already have a DevWorkspace / DevWorkspaceTemplates and so don't need to generate them.

Since this is intended for reuse in Che, I tested it there as well: I started a workspace from a private GitHub repository that contained the devfile for the Che dashboard. After bootstrapping was done, the resulting flattened DevWorkspace (what DWO is using to start the workspace internally) matches the DevWorkspace generated for the regular Che dashboard project:

--- dashboard.flat.devfile.yaml 2023-10-20 16:48:29.435215177 -0400
+++ bootstrapped.flat.devfile.yaml  2023-10-20 16:49:25.021803817 -0400
@@ -6,11 +6,8 @@
   controller.devfile.io/storage-type: per-user
   dw.metadata.annotations:
     che.eclipse.org/devfile-source: |
-      scm:
-        repo: https://github.com/eclipse-che/che-dashboard.git
-        fileName: devfile.yaml
       factory:
-        params: url=https://github.com/eclipse-che/che-dashboard.git
+        params: url=https://github.com/amisevsk/private-test-repo.git
 commands:
 - apply:
     component: che-code-injector
@@ -164,7 +161,7 @@
     cpuRequest: 1030m
     endpoints:
     - attributes:
-        controller.devfile.io/endpoint-url: https://eclipse-che.apps.<snip>.openshift.org/kube-admin/che-dashboard/3100/
+        controller.devfile.io/endpoint-url: https://eclipse-che.apps.<snip>.openshift.org/kube-admin/empty-i5oh/3100/
         cookiesAuthEnabled: true
         discoverable: false
         type: main
@@ -175,7 +172,7 @@
       secure: false
       targetPort: 3100
     - attributes:
-        controller.devfile.io/endpoint-url: http://kube-admin-che-dashboard-code-redirect-1.apps.<snip>.openshift.org/
+        controller.devfile.io/endpoint-url: http://kube-admin-empty-i5oh-code-redirect-1.apps.<snip>.openshift.org/
         discoverable: false
         urlRewriteSupported: false
       exposure: public
@@ -183,7 +180,7 @@
       protocol: http
       targetPort: 13131
     - attributes:
-        controller.devfile.io/endpoint-url: http://kube-admin-che-dashboard-code-redirect-2.apps.<snip>.openshift.org/
+        controller.devfile.io/endpoint-url: http://kube-admin-empty-i5oh-code-redirect-2.apps.<snip>.openshift.org/
         discoverable: false
         urlRewriteSupported: false
       exposure: public
@@ -191,7 +188,7 @@
       protocol: http
       targetPort: 13132
     - attributes:
-        controller.devfile.io/endpoint-url: http://kube-admin-che-dashboard-code-redirect-3.apps.<snip>.openshift.org/
+        controller.devfile.io/endpoint-url: http://kube-admin-empty-i5oh-code-redirect-3.apps.<snip>.openshift.org/
         discoverable: false
         urlRewriteSupported: false
       exposure: public
@@ -199,22 +196,20 @@
       protocol: http
       targetPort: 13133
     - attributes:
-        controller.devfile.io/endpoint-url: http://kube-admin-che-dashboard-local-server.apps.<snip>.openshift.org/
+        controller.devfile.io/endpoint-url: http://kube-admin-empty-i5oh-local-server.apps.<snip>.openshift.org/
       exposure: public
       name: local-server
       path: /
       protocol: http
       targetPort: 8080
     - attributes:
-        controller.devfile.io/endpoint-url: http://kube-admin-che-dashboard-bundle-analyzer.apps.<snip>.openshift.org/
+        controller.devfile.io/endpoint-url: http://kube-admin-empty-i5oh-bundle-analyzer.apps.<snip>.openshift.org/
       exposure: public
       name: bundle-analyzer
       path: /
       protocol: http
       targetPort: 8888
     env:
-    - name: KUBEDOCK_ENABLED
-      value: "true"
     - name: CHE_DASHBOARD_URL
       value: https://eclipse-che.apps.<snip>.openshift.org
     - name: CHE_PLUGIN_REGISTRY_URL
@@ -227,6 +222,8 @@
       value: OpenShift console
     - name: OPENVSX_REGISTRY_URL
       value: https://open-vsx.org
+    - name: KUBEDOCK_ENABLED
+      value: "true"
     image: quay.io/okurinny/dashboard.dev:skaffold
     memoryLimit: "11073741824"
     memoryRequest: 768Mi
@@ -244,6 +241,10 @@
 projects:
 - git:
     remotes:
+      origin: https://github.com/amisevsk/private-test-repo.git
+  name: test-project
+- git:
+    remotes:
       origin: https://github.com/eclipse-che/che-dashboard.git
   name: che-dashboard