redhat-developer / odo

odo - Developer-focused CLI for fast & iterative container-based application development on Podman and Kubernetes. Implementation of the open Devfile standard.
https://odo.dev
Apache License 2.0
792 stars 244 forks source link

Exec command for deploy does not support volumeMounts or mounts PROJECT_SOURCE #6658

Open valaparthvi opened 1 year ago

valaparthvi commented 1 year ago

/kind bug

What versions of software are you using?

Operating System: Darwin

Output of odo version: odo v3.7.0 (c3c360562)

How did you run odo exactly?

  1. git clone https://github.com/ehsavoie/numberguess.git
  2. git checkout odo
  3. odo deploy
devfile.yaml ```yaml schemaVersion: 2.2.0 metadata: name: java-wildfly-numberguess version: 2.0.0 website: https://wildfly.org displayName: WildFly Java Numberguess description: Upstream WildFly icon: https://design.jboss.org/wildfly/logo/final/wildfly_logomark.svg tags: ['Java', 'WildFly'] projectType: 'wildfly' language: 'java' attributes: controller.devfile.io/storage-type: ephemeral .vscode/extensions.json: | { "recommendations": [ "redhat.java", "vscjava.vscode-java-debug", "vscjava.vscode-java-test", "redhat.fabric8-analytics" ] } .che/che-theia-plugins.yaml: | - id: redhat/java components: - name: wildfly container: image: quay.io/wildfly/wildfly-s2i:latest memoryLimit: 1512Mi mountSources: true volumeMounts: - name: m2-repository path: /home/jboss/.m2/repository env: - name: GC_METASPACE_SIZE value: '96' - name: JAVA_OPTS value: '-Djava.security.egd=file:/dev/urandom' - name: MVN_ARGS_APPEND value: '-s /home/jboss/.m2/settings.xml -Dmaven.repo.local=/home/jboss/.m2/repository -Dcom.redhat.xpaas.repo.jbossorg' - name: DEBUG_PORT value: "5005" endpoints: - name: debug exposure: internal protocol: tcp targetPort: 5005 - name: 'http' protocol: http targetPort: 8080 exposure: public - name: 'management' targetPort: 9990 protocol: http exposure: internal - name: 'transaction' protocol: tcp targetPort: 4172 exposure: internal - name: tools container: image: quay.io/devfile/universal-developer-image:ubi8-latest memoryLimit: 1512Mi mountSources: true env: - name: OPENSHIFT_IMAGE_REGISTRY value: "image-registry.openshift-image-registry.svc:5000" - name: IMAGE value: "numberguess" - name: m2-repository volume: size: 3Gi commands: - id: init-server exec: label: "Init the server" component: wildfly commandLine: mvn ${MVN_ARGS_APPEND} -Pprovisioned-server -Dwildfly.javaOpts="-Djboss.node.name=numberguess -agentlib:jdwp=transport=dt_socket,address=0.0.0.0:${DEBUG_PORT},server=y,suspend=n" clean package org.wildfly.plugins:wildfly-maven-plugin:start workingDir: ${PROJECT_SOURCE} group: kind: run isDefault: false - id: debug exec: label: "Package and start the application in debug mode." component: wildfly commandLine: mvn ${MVN_ARGS_APPEND} -Pprovisioned-server -Dwildfly.javaOpts="-Djboss.node.name=numberguess -agentlib:jdwp=transport=dt_socket,address=0.0.0.0:${DEBUG_PORT},server=y,suspend=n" clean package org.wildfly.plugins:wildfly-maven-plugin:run workingDir: ${PROJECT_SOURCE} group: kind: debug isDefault: true - id: package exec: label: "Build the application." component: wildfly commandLine: mvn ${MVN_ARGS_APPEND} -Pprovisioned-server package workingDir: ${PROJECT_SOURCE} group: kind: build isDefault: true - id: deploy exec: label: "Build the application." component: wildfly commandLine: mvn ${MVN_ARGS_APPEND} -Popenshift -Dmaven.test.skip=true clean package workingDir: ${PROJECT_SOURCE} group: kind: run isDefault: true - id: outerloop-deploy composite: commands: - openshift-build - openshift-registry-login - build-runtime-image - push-image - deploy-image parallel: false label: "Build and deploy the application on Openshift." group: kind: deploy isDefault: true - id: openshift-build exec: label: "1 Build the application for OpenShift." component: wildfly commandLine: mvn ${MVN_ARGS_APPEND} -Popenshift -Dmaven.test.skip=true clean package workingDir: ${PROJECT_SOURCE} - id: openshift-registry-login exec: label: "2 Login into OpenShift registry." component: tools commandLine: podman login --tls-verify=false -u kubeadmin -p $(oc whoami -t) ${OPENSHIFT_IMAGE_REGISTRY} && oc login --token=$(oc whoami -t) workingDir: ${PROJECT_SOURCE} - id: build-runtime-image exec: label: "3 Build Runtime Image for Openshift." component: tools commandLine: "export OPENSHIFT_NS=$(oc project -q) && podman build -f Containerfile -t ${OPENSHIFT_IMAGE_REGISTRY}/${OPENSHIFT_NS}/${IMAGE} ." workingDir: ${PROJECT_SOURCE} - id: push-image exec: label: "4 Push Image into OpenShift registry." component: tools workingDir: ${PROJECT_SOURCE} commandLine: "export OPENSHIFT_NS=$(oc project -q) && podman tag ${OPENSHIFT_IMAGE_REGISTRY}/${OPENSHIFT_NS}/${IMAGE} ${OPENSHIFT_IMAGE_REGISTRY}/${OPENSHIFT_NS}/${IMAGE}:latest && podman push --tls-verify=false ${OPENSHIFT_IMAGE_REGISTRY}/${OPENSHIFT_NS}/${IMAGE}:latest" - id: deploy-image exec: label: "5 Deploy Image into OpenShift." component: tools workingDir: ${PROJECT_SOURCE} commandLine: "helm repo add wildfly https://docs.wildfly.org/wildfly-charts/ && oc set image-lookup ${IMAGE} && helm install ${IMAGE} -f charts/helm.yaml --set build.enabled=false --set image.name=${IMAGE} wildfly/wildfly" events: postStart: - 'init-server' ```
Containerfile ```Dockerfile FROM quay.io/wildfly/wildfly-runtime:latest COPY --chown=jboss:root target/server $JBOSS_HOME RUN chmod -R ug+rwX $JBOSS_HOME ```

Actual behavior

The command fails, but if you check the pod definition for job that runs exec command, there is no reference to volume mounts used by wildfly container component.

Expected behavior

If volume mounts were supported, it might be possible to successfully run odo deploy on the devfile.

Any logs, error output, etc?

/bin/sh: line 0: cd: /projects: No such file or directory
kadel commented 1 year ago

odo deploy is intended for outerloop runs. It is intentional that you don't have access to the source code inside the containers when doing odo deploy.

The following commands should be part of the Dockerfile/Containerfile

- id: openshift-build
    exec:
      label: "1 Build the application for OpenShift."
      component: wildfly
      commandLine: mvn ${MVN_ARGS_APPEND} -Popenshift -Dmaven.test.skip=true clean package
      workingDir: ${PROJECT_SOURCE}
  - id: openshift-registry-login
    exec:
      label: "2 Login into OpenShift registry."
      component: tools
      commandLine: podman login --tls-verify=false -u kubeadmin -p $(oc whoami -t) ${OPENSHIFT_IMAGE_REGISTRY} && oc login --token=$(oc whoami -t)
      workingDir: ${PROJECT_SOURCE}

I would do it like this.

Containerfile

FROM quay.io/wildfly/wildfly-runtime:latest
COPY --chown=jboss:root target/server $JBOSS_HOME
RUN chmod -R ug+rwX $JBOSS_HOME 

COPY . . 
RUN mvn ${MVN_ARGS_APPEND} -Popenshift -Dmaven.test.skip=true clean package

devfile.yaml

  - id: outerloop-deploy
    composite:
      commands:
            - build-runtime-image
            - deploy-image
      parallel: false
      label: "Build and deploy the application on Openshift."
      group:
          kind: deploy
          isDefault: true

  - id: build-runtimem-image
    image:
      imageName: ${OPENSHIFT_IMAGE_REGISTRY}/${OPENSHIFT_NS}/${IMAGE}
      dockerfile:
        uri: Containerfile
        buildContext: .

  - id: deploy-image
    exec:
      label: "5 Deploy Image into OpenShift."
      component: tools
      workingDir: ${PROJECT_SOURCE}
      commandLine: "helm repo add wildfly https://docs.wildfly.org/wildfly-charts/ && oc set image-lookup ${IMAGE} && helm install ${IMAGE} -f charts/helm.yaml --set build.enabled=false --set image.name=${IMAGE} wildfly/wildfly"

Not sure if this would currently work with odo; if not we need to identify what we are missing and enable this scenario.

ehsavoie commented 1 year ago

but that would defeat what I'm doing since that means that the build tools are part of the image which is not the case of the runtime-image. I guess I would have to use a multistep build ContainerFile which is quite different as the full build is not in the devfile but in the containerfile

kadel commented 1 year ago

I was checking again how this is defined in Devfile spec. And you are right that this is bug in the odo mountSources for container component has default value true and it should behave like this no matter where it is used (in run or deploy command). We will have to fix this in odo to do make it work as defined in Devfile spec.

But honestly, I still don't think this is a good way to do this. Multi-stage build would be a better approach. Storing the build instructions in a Containerfile has also its benefits as you will have a fully functional Containerfile so it can be used even by tools not understanding devfile.

valaparthvi commented 1 year ago

Grooming Call [ April 13, '23]

feloy commented 1 year ago

Here is another example where this feature would be needed. The deployment is done on GitHub pages and we need the sources to build the pages to deploy, but they are not mounted on the container.

https://github.com/feloy/devfile-builder

$ odo deploy
  __
 /  \__     Running the application in Deploy mode using devfile-builder Devfile
 \__/  \    Namespace: ns6671
 /  \__/    odo version: v3.10.0
 \__/

↪ Executing command:
 ✗  Executing command in container (command: deploy) [12s]
Execution output:
/bin/sh: line 0: cd: /projects: No such file or directory
 ✗  failed to execute (command: deploy)
github-actions[bot] commented 1 year ago

A friendly reminder that this issue had no activity for 90 days. Stale issues will be closed after an additional 30 days of inactivity.

github-actions[bot] commented 1 year ago

This issue was closed because it has been inactive for 30 days since being marked as stale.

valaparthvi commented 1 year ago

@rm3l Do you think this is still relevant?

rm3l commented 1 year ago

@rm3l Do you think this is still relevant?

Yes, that is still relevant. Let's reopen it. Thanks.

/remove-lifecycle rotten /remove-lifecycle stale /reopen

openshift-ci[bot] commented 1 year ago

@rm3l: Reopened this issue.

In response to [this](https://github.com/redhat-developer/odo/issues/6658#issuecomment-1704883518): >> @rm3l Do you think this is still relevant? > >Yes, that is still relevant. Let's reopen it. Thanks. > >/remove-lifecycle rotten >/remove-lifecycle stale >/reopen Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
github-actions[bot] commented 11 months ago

A friendly reminder that this issue had no activity for 90 days. Stale issues will be closed after an additional 30 days of inactivity.