nickboldt commented 2 years ago

What does this PR do?

fix: CRW-2509 split yq install into runtime-from step (install pyyaml rpm, fetch wheels) and runtime-setup step (install from wheels); this also move some ubi-brew override stuff up into Theia so we have less deviation from upstream (and more information about installed rpms)

Change-Id: I6c32601122c6d305d39c4968426691eb273734da Signed-off-by: nickboldt nboldt@redhat.com

Screenshot/screencast of this PR

N/A

What issues does this PR fix or reference?

https://issues.redhat.com/browse/CRW-2509

How to test this PR?

N/A

PR Checklist

As the author of this Pull Request I made sure that:

Reviewers

Reviewers, please comment how you tested the PR when approving it.

nickboldt commented 2 years ago

With these changes we can do this in downstream:

And the runtime-from and runtime-setup files downstream become the same as upstream.

codecov[bot] commented 2 years ago

Codecov Report

Merging #1265 (2481dd2) into main (c299f59) will increase coverage by 2.97%. The diff coverage is 28.66%.

@@            Coverage Diff             @@
##             main    #1265      +/-   ##
==========================================
+ Coverage   32.78%   35.76%   +2.97%     
==========================================
  Files         290      324      +34     
  Lines        9885    10460     +575     
  Branches     1457     1425      -32     
==========================================
+ Hits         3241     3741     +500     
- Misses       6641     6715      +74     
- Partials        3        4       +1

Impacted Files	Coverage Δ
...theia-about/src/browser/about-che-theia-dialog.tsx	`0.00% <0.00%> (ø)`
...credentials/src/browser/che-credentials-service.ts	`0.00% <0.00%> (ø)`
...entials/src/browser/credentials-frontend-module.ts	`0.00% <0.00%> (ø)`
...eia-credentials/src/common/credentials-protocol.ts	`0.00% <0.00%> (ø)`
...eia-credentials/src/node/che-credentials-server.ts	`0.00% <0.00%> (ø)`
...s/src/node/che-theia-credentials-backend-module.ts	`0.00% <0.00%> (ø)`
...ashboard/src/browser/che-theia-dashboard-module.ts	`0.00% <0.00%> (ø)`
...ia-dashboard/src/browser/theia-dashboard-client.ts	`0.00% <0.00%> (ø)`
...rowser/src/browser/che-mini-browser-environment.ts	`0.00% <0.00%> (ø)`
...ia-plugin-remote/src/node/hosted-plugin-service.ts	`0.00% <0.00%> (ø)`
... and 257 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 8a48c2f...2481dd2. Read the comment docs.

nickboldt commented 2 years ago

Downstream generated dockerfile should look like this.

Passed in brew here: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=41550826

# Copyright (c) 2019-21 Red Hat, Inc.
# This program and the accompanying materials are made
# available under the terms of the Eclipse Public License 2.0
# which is available at https://www.eclipse.org/legal/epl-2.0/
#
# SPDX-License-Identifier: EPL-2.0
#
# Contributors:
#   Red Hat, Inc. - initial API and implementation

FROM registry-proxy.engineering.redhat.com/rh-osbs/codeready-workspaces-theia-rhel8:2.14 as builder
ARG NEXE_SHA1=0f0869b292f1d7b68ba6e170d628de68a10c009f

WORKDIR /home/theia

# Apply node libs installed globally to the PATH
ENV PATH=${HOME}/.yarn/bin:${PATH}

# setup extra stuff
USER root
ENV YARN_FLAGS="--offline"
ENV NEXE_FLAGS="--target 'alpine-x64-12' --temp /tmp/nexe-cache"
COPY asset-theia-endpoint-runtime-pre-assembly-nexe-cache-*.tar.gz asset-theia-endpoint-runtime-binary-yarn-*.tar.gz asset-node-src.tar.gz /tmp/
RUN tar xzf /tmp/asset-theia-endpoint-runtime-binary-yarn-$(uname -m).tar.gz -C / && rm -f /tmp/asset-theia-endpoint-runtime-binary-yarn-*.tar.gz && \
    export NODE_VERSION=$(node --version | sed -s 's/v//') && mkdir -p "/home/theia/.nexe/${NODE_VERSION}" && \
    tar zxf /tmp/asset-node-src.tar.gz --strip-components=1 -C "/home/theia/.nexe/${NODE_VERSION}" && \
    rm -f /tmp/asset-node-src.tar.gz && \
    tar zxf /tmp/asset-theia-endpoint-runtime-pre-assembly-nexe-cache-$(uname -m).tar.gz -C "/tmp/" && \
    rm -f /tmp/asset-theia-endpoint-runtime-pre-assembly-nexe-cache-*.tar.gz 

RUN yum install -y git make cmake gcc gcc-c++ python2 automake autoconf which glibc-devel && \
    yum -y clean all && rm -rf /var/cache/yum && ln -s /usr/bin/python2 /usr/bin/python
# setup nexe
# install specific nexe
COPY asset-theia-endpoint-runtime-pre-assembly-nexe-*.tar.gz /tmp/
RUN tar zxf /tmp/asset-theia-endpoint-runtime-pre-assembly-nexe-$(uname -m).tar.gz -C "/tmp/" && \
    rm -f /tmp/asset-theia-endpoint-runtime-pre-assembly-nexe-*.tar.gz 
# Change back to root folder
WORKDIR /home/theia

RUN /tmp/nexe/index.js -v && \
    # Build remote binary with node runtime 12.x and che-theia node dependencies. nexe icludes to the binary only
    # necessary dependencies.
    eval /tmp/nexe/index.js -i node_modules/@eclipse-che/theia-remote/lib/node/plugin-remote.js ${NEXE_FLAGS} -o ${HOME}/plugin-remote-endpoint

# Light image without node. We include remote binary to this image.
# https://access.redhat.com/containers/?tab=tags#/registry.access.redhat.com/ubi8-minimal
FROM ubi8-minimal:8.5-204 as runtime
USER 0
RUN microdnf -y install yum python38 python38-pyyaml jq && python3 --version && \
    yum -y -q update && \
    yum -y -q clean all && rm -rf /var/cache/yum && \
    echo "Installed Packages" && rpm -qa | sort -V && echo "End Of Installed Packages"

COPY *.whl /tmp

ENV SUMMARY="Red Hat CodeReady Workspaces - theia-endpoint container" \
    DESCRIPTION="Red Hat CodeReady Workspaces - theia-endpoint container" \
    PRODNAME="codeready-workspaces" \
    COMPNAME="theia-endpoint-rhel8" 

LABEL summary="$SUMMARY" \
      description="$DESCRIPTION" \
      io.k8s.description="$DESCRIPTION" \
      io.k8s.display-name="$DESCRIPTION" \
      io.openshift.tags="$PRODNAME,$COMPNAME" \
      com.redhat.component="$PRODNAME-$COMPNAME-container" \
      name="$PRODNAME/$COMPNAME" \
      version="2.14" \
      license="EPLv2" \
      maintainer="Nick Boldt <nboldt@redhat.com>" \
      io.openshift.expose-services="" \
      usage=""

# Setup extra stuff

# install yq from local wheels we fetched earlier 
RUN pip3 --version && pip3 install /tmp/*.whl && yq --version && rm -fr /tmp/*.whl 

COPY --from=builder /home/theia/plugin-remote-endpoint /plugin-remote-endpoint

COPY src/entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

che-bot commented 2 years ago

:white_check_mark: E2E Happy path tests succeed :tada:

See Details

- [Jenkins job](https://main-jenkins-csb-crwqe.apps.ocp4.prod.psi.redhat.com/blue/organizations/jenkins/Che%2Fe2e%2Fminikube%2Fpr-check%2Fche-theia-pr-check/detail/che-theia-pr-check/612) - [test report](https://main-jenkins-csb-crwqe.apps.ocp4.prod.psi.redhat.com/job/Che/job/e2e/job/minikube/job/pr-check/job/che-theia-pr-check/612/artifact/che/tests/e2e/report/) - [logs and configs](https://main-jenkins-csb-crwqe.apps.ocp4.prod.psi.redhat.com/job/Che/job/e2e/job/minikube/job/pr-check/job/che-theia-pr-check/612/artifact/logs-and-configs/) - [Happy path tests DevFile](https://raw.githubusercontent.com/chepullreq4/pr-check-files/master/che-theia/pr-1265/happy-path-workspace.yaml) - images: | name | link| |---|---| | che-theia | quay.io/crw_pr/che-theia:1265| | che-theia-endpoint-runtime-binary | quay.io/crw_pr/che-theia-endpoint-runtime-binary:1265| **Test product**: - tested with Eclipse Che on K8S (minikube v1.22.0) - [E2E test scenario](https://github.com/eclipse/che/blob/main/tests/e2e/tests/e2e_happy_path/HappyPath.spec.ts) - [E2E test pipeline source code](https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/blob/master/jobs/Che/e2e/minikube/pr-check/che-theia-pr-check.jenkinsfile)

Use comment "[crw-ci-test]" to rerun happy path E2E test.
Use comment "[crw-ci-test --rebuild]" to re-build the images and rerun happy path E2E test.

Eclipse Che QE channel: https://mattermost.eclipse.org/eclipse/channels/eclipse-che-qe

eclipse-che / che-theia

fix: CRW-2509 theia-endpoint: split yq install into 2 steps: install pyyaml, fetch wheels; install from wheels; also remove some ubi-brew overrides so downstream closer to upstream #1265

What does this PR do?

Screenshot/screencast of this PR

What issues does this PR fix or reference?

How to test this PR?

PR Checklist

Reviewers

Codecov Report