search

nodeshift-archived / ubi8-s2i-web-app

Apache License 2.0
26 stars 28 forks source link

OCP 4 fails to build (manifest unknown) #38

Closed cfchase closed 5 years ago

cfchase commented 5 years ago

Trying use a chained build which works on on OpenShift 3.11, but fails on OpenShift 4.0. To reproduce a simple case, I used the project shown in lholmquist/react-web-app and followed the instructions.

find . | grep openshiftio | grep application | xargs -n 1 oc apply -f
oc new-app --template react-web-app -p SOURCE_REPOSITORY_URL=https://github.com/lholmquist/react-web-app

It looks like it's trying to pull its own target image before anything exists yet. The result on OpenShift 4 is:

$ oc logs react-web-app-builder-1-build

Caching blobs under "/var/cache/blobs".
Pulling docker://nodeshift/centos7-s2i-web-app@sha256:5a0ea5fbaebc29f3772042b243d5a70cb90c0b74720148c9d5287adc4a3b3038
Getting image source signatures
Copying blob sha256:aeb7866da422acc7e93dcf7323f38d7646f6269af33bcdb6647f2094fc4b3bf7
Copying blob sha256:9291e350be08c93d4e48286db9e72d9c74c7cba5778729d167220c664656d314
Copying blob sha256:4d3620628f7cc3d26ed80469f503f521a9939c2aaac68ec1a1bb4076d92b77e8
Copying blob sha256:94d526a435e926d08aaf39d22731d1d58935203409978a2a7eb170fb76a9d4d8
Copying blob sha256:2603cb6c0d494298779d11c9f9888824a121946162ad28397377d47738b15f5e
Copying blob sha256:27d649e09bdba6830975348259572dddf16bf870b5e685f311008fe117d40baf
Copying blob sha256:d5957af9f715def3f63d1dae0583783e9fbd541a47dfddfe833d60f15283e004
Copying blob sha256:8a88affce9a736bb5e77a2dac18218488ba692a01e8a41f27d3e3b9ebf8ace22
Copying blob sha256:74fd737edd185d29986788f4045004c58c7e7ac4fff1008d3c5d326ebb001abe
Copying config sha256:5e2129220bb621b9254ee6b5fbe7e2eee59e7bc1fc05846d5287748afe854351
Writing manifest to image destination
Storing signatures
5e2129220bb621b9254ee6b5fbe7e2eee59e7bc1fc05846d5287748afe854351
Generating dockerfile with builder image nodeshift/centos7-s2i-web-app@sha256:5a0ea5fbaebc29f3772042b243d5a70cb90c0b74720148c9d5287adc4a3b3038
STEP 1: FROM image-registry.openshift-image-registry.svc:5000/cchase/react-web-app-builder:latest AS cached
error: build error: error creating build container: Error determining manifest MIME type for docker://image-registry.openshift-image-registry.svc:5000/cchase/react-web-app-builder:latest: Error reading manifest latest in image-registry.openshift-image-registry.svc:5000/cchase/react-web-app-builder: manifest unknown: manifest unknown

My version of openshift:

$ oc version
oc v4.0.0-0.139.0
kubernetes v1.11.0+c69f926354
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://***:***
kubernetes v1.12.4+f39ab668d3

On the FIRST build in OpenShift, there's a similar error ("pulling image error"), but it doesn't fail the build. OpenShift 3.11:

Cloning "https://github.com/lholmquist/react-web-app.git " ...
    Commit: 244e2ff913b0aa151a00c53f0eb6393dcf3bffe9 (chore: update the template to use the nodeshift namespaced builder image (#2))
    Author: Lucas Holmquist <lholmqui@redhat.com>
    Date:   Fri Oct 26 10:02:39 2018 -0400
Using nodeshift/centos7-s2i-web-app@sha256:5a0ea5fbaebc29f3772042b243d5a70cb90c0b74720148c9d5287adc4a3b3038 as the s2i builder image
Pulling image "172.30.1.1:5000/rwa/react-web-app-builder:latest" ...
pulling image error : Error: image rwa/react-web-app-builder:latest not found

# ... more successful build log stuff ...
Push successful

If I can help with this any way I can, let me know. I haven't written my own s2i builder image before, so I'm not clear on why it would need to pull an image before it's been builts.

lholmquist commented 5 years ago

I think the error that you see on the 3.11 example, is because the application is set to use incremental builds, https://github.com/lholmquist/react-web-app/blob/master/.openshiftio/application.yaml#L92. So I think when you do a second build it might go away. Can you try removing that line and see what happens? maybe try that with the openshift 4 tests

Not sure this is a problem with the image, though. But i haven't tested on Openshift 4 yet

cfchase commented 5 years ago

Thanks for the quick response. Progress!

New error now, though

Caching blobs under "/var/cache/blobs".
Pulling docker://nodeshift/centos7-s2i-web-app@sha256:5a0ea5fbaebc29f3772042b243d5a70cb90c0b74720148c9d5287adc4a3b3038
Getting image source signatures
Copying blob sha256:aeb7866da422acc7e93dcf7323f38d7646f6269af33bcdb6647f2094fc4b3bf7
Copying blob sha256:9291e350be08c93d4e48286db9e72d9c74c7cba5778729d167220c664656d314
Copying blob sha256:4d3620628f7cc3d26ed80469f503f521a9939c2aaac68ec1a1bb4076d92b77e8
Copying blob sha256:94d526a435e926d08aaf39d22731d1d58935203409978a2a7eb170fb76a9d4d8
Copying blob sha256:2603cb6c0d494298779d11c9f9888824a121946162ad28397377d47738b15f5e
Copying blob sha256:27d649e09bdba6830975348259572dddf16bf870b5e685f311008fe117d40baf
Copying blob sha256:d5957af9f715def3f63d1dae0583783e9fbd541a47dfddfe833d60f15283e004
Copying blob sha256:8a88affce9a736bb5e77a2dac18218488ba692a01e8a41f27d3e3b9ebf8ace22
Copying blob sha256:74fd737edd185d29986788f4045004c58c7e7ac4fff1008d3c5d326ebb001abe
Copying config sha256:5e2129220bb621b9254ee6b5fbe7e2eee59e7bc1fc05846d5287748afe854351
Writing manifest to image destination
Storing signatures
5e2129220bb621b9254ee6b5fbe7e2eee59e7bc1fc05846d5287748afe854351
Generating dockerfile with builder image nodeshift/centos7-s2i-web-app@sha256:5a0ea5fbaebc29f3772042b243d5a70cb90c0b74720148c9d5287adc4a3b3038
STEP 1: FROM nodeshift/centos7-s2i-web-app@sha256:5a0ea5fbaebc29f3772042b243d5a70cb90c0b74720148c9d5287adc4a3b3038
STEP 2: LABEL "io.openshift.build.commit.author"="Lucas Holmquist <lholmqui@redhat.com>" "io.openshift.build.commit.date"="Fri Oct 26 10:02:39 2018 -0400" "io.openshift.build.commit.id"="244e2ff913b0aa151a00c53f0eb6393dcf3bffe9" "io.openshift.build.commit.ref"="master" "io.openshift.build.commit.message"="chore: update the template to use the nodeshift namespaced builder image (#2)" "io.openshift.build.source-location"="https://github.com/lholmquist/react-web-app" "io.openshift.build.image"="nodeshift/centos7-s2i-web-app@sha256:5a0ea5fbaebc29f3772042b243d5a70cb90c0b74720148c9d5287adc4a3b3038"
STEP 3: ENV OPENSHIFT_BUILD_NAME="react-web-app-builder-1" OPENSHIFT_BUILD_NAMESPACE="web-game-demo" OPENSHIFT_BUILD_SOURCE="https://github.com/lholmquist/react-web-app" OPENSHIFT_BUILD_REFERENCE="master" OPENSHIFT_BUILD_COMMIT="244e2ff913b0aa151a00c53f0eb6393dcf3bffe9" OUTPUT_DIR="build"
STEP 4: USER root
STEP 5: COPY upload/src /tmp/src
STEP 6: RUN chown -R 1001:0 /tmp/src
time="2019-02-27T20:15:36Z" level=warning msg="pkg/chroot: error unmounting \"/tmp/buildah683737400/mnt/rootfs\": error checking if \"/tmp/buildah683737400/mnt/rootfs/sys/fs/cgroup/perf_event\" is mounted: no such file or directory"
time="2019-02-27T20:15:36Z" level=warning msg="pkg/bind: error unmounting \"/tmp/buildah683737400/mnt/rootfs\": error checking if \"/tmp/buildah683737400/mnt/rootfs/sys/fs/cgroup/perf_event\" is mounted: no such file or directory"
STEP 7: USER 1001
STEP 8: RUN /usr/libexec/s2i/assemble
ls: cannot access /tmp/src/node_modules: No such file or directory
---> Installing application source
---> Building your Web Application from source
Current git config
url.https://github.com.insteadof=git@github.com:
url.https://.insteadof=ssh://
url.https://github.com.insteadof=ssh://git@github.com
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.url=https://github.com/lholmquist/react-web-app
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.master.remote=origin
branch.master.merge=refs/heads/master
---> Installing dependencies
---> Using 'npm install'

subprocess exited with status 1
subprocess exited with status 1
error: build error: error building at step {Env:[OUTPUT_DIR=build OPENSHIFT_BUILD_COMMIT=244e2ff913b0aa151a00c53f0eb6393dcf3bffe9 OPENSHIFT_BUILD_REFERENCE=master OPENSHIFT_BUILD_SOURCE=https://github.com/lholmquist/react-web-app OPENSHIFT_BUILD_NAMESPACE=web-game-demo OPENSHIFT_BUILD_NAME=react-web-app-builder-1 PATH=/opt/app-root/src/node_modules/.bin/:/opt/app-root/src/.npm-global/bin/:/opt/app-root/src/bin:/opt/app-root/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin SUMMARY=Platform for building Modern Web Applications that use Node.js DESCRIPTION=Web Application building with Node.js STI_SCRIPTS_URL=image:///usr/libexec/s2i STI_SCRIPTS_PATH=/usr/libexec/s2i APP_ROOT=/opt/app-root HOME=/opt/app-root/src BASH_ENV=/opt/app-root/etc/scl_enable ENV=/opt/app-root/etc/scl_enable PROMPT_COMMAND=. /opt/app-root/etc/scl_enable NODEJS_SCL=rh-nodejs8 NPM_RUN= NODE_VERSION=10.15.1 NPM_VERSION=6.4.1 NODE_LTS=false NPM_CONFIG_LOGLEVEL=info NPM_CONFIG_PREFIX=/opt/app-root/src/.npm-global NPM_CONFIG_TARBALL=/usr/share/node/node-v10.15.1-headers.tar.gz DEBUG_PORT=5858 OUTPUT_DIR=build NPM_BUILD=npm run build DEPLOY_PORT=8080       OPENSHIFT_BUILD_NAME=react-web-app-builder-1 OPENSHIFT_BUILD_NAMESPACE=web-game-demo OPENSHIFT_BUILD_SOURCE=https://github.com/lholmquist/react-web-app OPENSHIFT_BUILD_REFERENCE=master OPENSHIFT_BUILD_COMMIT=244e2ff913b0aa151a00c53f0eb6393dcf3bffe9 OUTPUT_DIR=build] Command:run Args:[/usr/libexec/s2i/assemble] Flags:[] Attrs:map[] Message:RUN /usr/libexec/s2i/assemble Original:RUN /usr/libexec/s2i/assemble}: exit status 1
lholmquist commented 5 years ago

I've finally setup openshift 4, and i'm getting the same thing. Will investigate

lholmquist commented 5 years ago

i actually did this to create the app: oc new-app nodeshift/centos7-s2i-web-app:latest~https://github.com/lholmquist/react-web-app

lholmquist commented 5 years ago

i tried using yarn instead of npm by setting YARN_ENABLED=true and everything worked fine. So that narrows things down now to something happening during the build when using npm. I think this line is the culprit, https://github.com/nodeshift/centos7-s2i-web-app/blob/master/s2i/assemble#L58, but not really sure why

lholmquist commented 5 years ago

I updated the npm install to do npm install -d to get a bit more info

for this error:

....
npm timing action:finalize Completed in 1416ms
npm timing npm Completed in 30681ms
npm ERR! cb() never called!

npm ERR! This is an error with npm itself. Please report this error at:
npm ERR!     <https://npm.community>

So it isn't even making it to the npm run build stuff yet.

I even tried removing the package-lock.json to see if that was it, but nope

lholmquist commented 5 years ago

I tried running just the nodejs image with the rest-http-booster

oc new-app nodeshift/centos7-s2i-nodejs:latest~https://github.com/nodeshift/nodejs-rest-http, which worked fine.

So then i used that nodejs image with the react-app, and i got the same error as before. so it looks like something is happening during the install

lholmquist commented 5 years ago

@cfchase i haven't had time to look at his since the last updated i made, but i'm planning on getting back into it

cfchase commented 5 years ago

Thanks for your efforts.

lholmquist commented 5 years ago

little update here. I tried running this with the node 8.x version(which we stopped making and lives at bucharestgold/centos7-s2i-web-app)

That is using npm 5.6.0, not sure why it fails on node 10.

the hunt continues

cfchase commented 5 years ago

Out of curiosity for a workaround, is there any way to run these chained s2i builds locally using the command line? 

s2i build ${REPO} nodeshift/centos7-s2i-web-app:10.x ${BUILDER_IMAGE}
s2i build ${BUILDER_IMAGE_ARTIFACT} centos/nginx-112-centos7 ${RUNTIME_IMAGE}
lholmquist commented 5 years ago

@cfchase i'm not sure how it would work exactly if running things locally, but you could just run the "npm build" locally, then tell the nginx image to use the resulting build folder.

it sort of defeats the purpose though

lholmquist commented 5 years ago

Another update: I used npm 5.6.0 with node 10 and i was able to get things to work. so perhaps it is an issue with the 6.4.x(?) npm version?

lholmquist commented 5 years ago

i tried with the 11.x version of this image, and that worked, since it is using npm 6.7.0

lholmquist commented 5 years ago

Some updates:

One of the tests i did, was to get a generic(using the web app builder image) pod deployed, then entered the terminal, and ran the "create-react-app" process, and then doing an npm install on that new project. This surprisingly didn't fail, which I thought was odd.

After doing a little bit of digging in the npm community forums, i stumbled across this thread, https://npm.community/t/crash-npm-err-cb-never-called/858/62, which basically said to try the latest version of npm since there had been some changes to how errors were reported.

I tried then running the image with npm 6.6.0 and instead of getting a generic error, i got an error about "EMFILE: too many open files", which relates to the ulimit value. I changed the image to print out that value(ulimit -n) when the build ran, and it was set to "1024", but in the deploy pod, it was set to "1048576". That 1048576 number was what the "ulimit -n" was set to in my 3.11 minishift for the build phase.

So it seems we need to update this value, which can't actually be done during our assemble script. I'll need to ask someone from the openshift team, if and how it is possible to increase that limit

lholmquist commented 5 years ago

closing since i think this is resolved with a newer version of npm that is out