Using `devspace dev` on OpenShift

[akosma]

What happened?

Following the Quickstart Guide with an OpenShift 4.9 cluster, the command devspace dev does not work as expected. I'm working on this ticket to document how to use DevSpace with APPUiO Cloud, an OpenShift 4.9 based platform from Switzerland.

What did you expect to happen instead?

It should have worked so as to continue with further steps of the tutorial.

How can we reproduce the bug? (as minimally and precisely as possible)

My devspace.yaml:

version: v1beta11

# `vars` specifies variables which may be used as ${VAR_NAME} in devspace.yaml
vars:
- name: IMAGE
  value: akosma/app

# `deployments` tells DevSpace how to deploy this project
deployments:
- name: devspace-quickstart-golang
  # This deployment uses `helm` but you can also define `kubectl` deployments or kustomizations
  helm:
    # We are deploying the so-called Component Chart: https://devspace.sh/component-chart/docs
    componentChart: true
    # Under `values` we can define the values for this Helm chart used during `helm install/upgrade`
    # You may also use `valuesFiles` to load values from files, e.g. valuesFiles: ["values.yaml"]
    values:
      containers:
      - image: ${IMAGE} # Use the value of our `${IMAGE}` variable here (see vars above)
      service:
        ports:
        - port: 8080

# `dev` only applies when you run `devspace dev`
dev:
  # `dev.ports` specifies all ports that should be forwarded while `devspace dev` is running
  # Port-forwarding lets you access your application via localhost on your local machine
  ports:
  - imageSelector: ${IMAGE} # Select the Pod that runs our `${IMAGE}`
    forward:
    - port: 8080

  # `dev.open` tells DevSpace to open certain URLs as soon as they return HTTP status 200
  # Since we configured port-forwarding, we can use a localhost address here to access our application
  open:
  - url: http://localhost:8080

  # `dev.sync` configures a file sync between our Pods in k8s and your local project files
  sync:
  - imageSelector: ${IMAGE} # Select the Pod that runs our `${IMAGE}`
    # `excludePaths` option expects an array of strings with paths that should not be synchronized between the
    # local filesystem and the remote container filesystem. It uses the same syntax as `.gitignore`.
    excludePaths:
    - .git/
    uploadExcludePaths:
    - Dockerfile

  # `dev.terminal` tells DevSpace to open a terminal as a last step during `devspace dev`
  terminal:
    imageSelector: ${IMAGE} # Select the Pod that runs our `${IMAGE}`
    # With this optional `command` we can tell DevSpace to run a script when opening the terminal
    # This is often useful to display help info for new users or perform initial tasks (e.g. installing dependencies)
    # DevSpace has generated an example ./devspace_start.sh file in your local project - Feel free to customize it!
    command:
    - ./devspace_start.sh

  # Since our Helm charts and manifests deployments are often optimized for production,
  # DevSpace let's you swap out Pods dynamically to get a better dev environment
  replacePods:
  - imageSelector: ${IMAGE} # Select the Pod that runs our `${IMAGE}`
    # Since the `${IMAGE}` used to start our main application pod may be distroless or not have any dev tooling, let's replace it with a dev-optimized image
    # DevSpace provides a sample image here but you can use any image for your specific needs
    replaceImage: loftsh/go:latest
    # Besides replacing the container image, let's also apply some patches to the `spec` of our Pod
    # We are overwriting `command` + `args` for the first container in our selected Pod, so it starts with `sleep 9999999`
    # Using `sleep 9999999` as PID 1 (instead of the regular ENTRYPOINT), allows you to start the application manually
    patches:
    - op: replace
      path: spec.containers[0].command
      value:
      - sleep
    - op: replace
      path: spec.containers[0].args
      value:
      - "9999999"
    - op: remove
      path: spec.containers[0].securityContext

# `profiles` lets you modify the config above for different environments (e.g. dev vs production)
profiles:
  # This profile is called `production` and you can use it for example using: devspace deploy -p production
  # We generally recommend using the base config without any profiles as optimized for development (e.g. image build+push is disabled)
- name: production
# This profile adds our image to the config so that DevSpace will build, tag and push our image before the deployment
  merge:
    images:
      app:
        image: ${IMAGE} # Use the value of our `${IMAGE}` variable here (see vars above)
        dockerfile: ./Dockerfile

Local Environment:

• DevSpace Version: devspace version 5.18.3
• Operating System: linux (Ubuntu 20.04)
• ARCH of the OS: AMD64

Kubernetes Cluster:

• Cloud Provider: other => https://appuio.cloud/
• Kubernetes Version: (output of oc version) Client Version: 4.8.0-202107292232.p0.git.1077b05.assembly.stream-1077b05 Kubernetes Version: v1.22.3+e790d7f

(output of kubectl version) Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.2", GitCommit:"9d142434e3af351a628bffee3939e64c681afa4d", GitTreeState:"clean", BuildDate:"2022-01-19T17:35:46Z", GoVersion:"go1.17.5", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.3+e790d7f", GitCommit:"3a0f2c90b43e6cffd07f57b5b78dd9f083e47ee2", GitTreeState:"clean", BuildDate:"2021-12-14T02:10:38Z", GoVersion:"go1.16.6", Compiler:"gc", Platform:"linux/amd64"}

Anything else we need to know?

devspace deploy -p production works without problem. The application is deployed as expected.

This is the output of the devspace dev command:

$ devspace use namespace devspace-test                                                                                          
[done] √ Successfully set default namespace to 'devspace-test'
$ devspace dev   
[info]   Using namespace 'devspace-test'
[info]   Using kube context 'devspace-test/api-exoscale-ch-gva-2-0-appuio-cloud:6443/roil-akosmaczewski1'
[info]   Execute 'helm upgrade devspace-quickstart-golang --namespace devspace-test --values /tmp/1423452300 --install /home/akosma/.devspace/component-chart/component-chart-0.8.4.tgz --kube-context devspace-test/api-exoscale-ch-gva-2-0-appuio-cloud:6443/roil-akosmaczewski1'
[done] √ Deployed helm chart (Release revision: 2)                                                                                         
[done] √ Successfully deployed devspace-quickstart-golang with helm                                                                        
[0:replacePod] Try to find replaced pod...
[0:replacePod] Try to find replaceable pod...
[0:replacePod] Replacing Pod devspace-test/devspace-quickstart-golang-6b76c9d645-gf75b...
[0:replacePod] Scaled down Deployment devspace-test/devspace-quickstart-golang
[0:replacePod] Waiting for Pod devspace-quickstart-golang-6b76c9d645-gf75b to get terminated...
[0:replacePod] Successfully replaced pod devspace-test/devspace-quickstart-golang-6b76c9d645-gf75b

#########################################################
[info]   DevSpace UI available at: http://localhost:8090
#########################################################

[0:ports] Port-Forwarding: Waiting for containers to start...
[0:sync] Waiting for containers to start...
[0:sync] Starting sync...
[0:ports] Port forwarding started on 8080:8080 (devspace-test/devspace-quickstart-golang-6b76c9d645-gf75b-66zpc)
[0:sync] Sync started on /home/akosma/Desktop/devspace-quickstart-golang <-> . (Pod: devspace-test/devspace-quickstart-golang-6b76c9d645-gf75b-66zpc)
[0:sync] Waiting for initial sync to complete
[fatal]  start sync: initial sync: upstream: apply changes: apply creates: upload archive: after upload: rpc error: code = Unknown desc = untar all: decompress: create /app/.dockerignore: open /app/.dockerignore: permission denied

/kind bug

[FabianKramm]

@akosma thanks for creating this issue! Looks like this is a permission problem within the development container. You could erase the replacePods[*].image config option and try to develop directly in your akosma/app container which might have the correct permissions defined. In general, DevSpace will need read / write permissions to the folders you want to sync between your local and remote environment, so make sure that the default user has access within the container to those folders.

[akosma]

Thanks a lot for the tip @FabianKramm ! I commented out that part, and tried again, but without success:

$ devspace dev     
[info]   Using namespace 'devspace-test'
[info]   Using kube context 'devspace-test/api-exoscale-ch-gva-2-0-appuio-cloud:6443/roil-akosmaczewski1'
[info]   Execute 'helm upgrade devspace-quickstart-golang --namespace devspace-test --values /tmp/2282423987 --install /home/akosma/.devspace/component-chart/component-chart-0.8.4.tgz --kube-context devspace-test/api-exoscale-ch-gva-2-0-appuio-cloud:6443/roil-akosmaczewski1'
[done] √ Deployed helm chart (Release revision: 5)                                                                                         
[done] √ Successfully deployed devspace-quickstart-golang with helm                                                                        

#########################################################
[info]   DevSpace UI available at: http://localhost:8090
#########################################################

[0:sync] Waiting for containers to start...
[0:ports] Port-Forwarding: Waiting for containers to start...
[0:sync] Starting sync...
[0:ports] Port forwarding started on 8080:8080 (devspace-test/devspace-quickstart-golang-d56988855-97tjp)
[fatal]  start sync: start sync: error executing tar: time="2022-02-14T13:43:08Z" level=error msg="exec failed: container_linux.go:380: starting container process caused: exec: \"tar\": executable file not found in $PATH"
: command terminated with exit code 1

[FabianKramm]

@akosma seems like your container has no tar executable in path. DevSpace usually requires some sort of development container in which paths are synced from the local machine and build chains are available to build the executable. Usually this can be accomplished by using multi stage builds with docker and then using an intermediate container as development container.

Since you are working with OpenShift and non root containers are default there, you will also need to make sure that you have sufficient permissions in that development container to change the paths you will need access to while developing.

[akosma]

Thanks for your help. We're going to stop evaluating devspace for the moment. I suggest to add an FAQ entry about OpenShift to the project, since devspace does not support it off-the-box at the moment. Thanks!