pipex commented 2 years ago

Expected Behavior

The build succeeds

Actual Behavior

Build fails to find commands in PATH

Steps to Reproduce the Problem

Create the following Dockerfile.template

FROM balenalib/%%BALENA_MACHINE_NAME%%-alpine

RUN echo "HELLO WORLD" | cat

ENV PATH="$PATH"

RUN echo "HELLO WORLD" | cat

Run an emulated build (tested on ubuntu 18.04)

DEBUG=1 balena build --debug --deviceType raspberrypi3 --arch armv7hf --emulated
[debug] original argv0="/usr/local/lib/balena-cli/bin/node" argv=[/usr/local/lib/balena-cli/bin/node,/usr/local/lib/balena-cli/bin/run,build,--debug,--deviceType,raspberrypi3,--arch,armv7hf,--emulated] length=9
[debug] new argv=[/usr/local/lib/balena-cli/bin/node,/usr/local/lib/balena-cli/bin/run,build,--deviceType,raspberrypi3,--arch,armv7hf,--emulated] length=8
[debug] Deprecation check: 6.84444 days since last npm registry query for next major version release date.
[debug] Will not query the registry again until at least 7 days have passed.
[Debug]   Parsing input...
[Debug]   Loading project...
[Debug]   Resolving project...
[Info]    No "docker-compose.yml" file found at "/Users/flalanne/Development/balena/support/build-issues/test"
[Info]    Creating default composition with source: "/Users/flalanne/Development/balena/support/build-issues/test"
[Debug]   Creating project...
[Build]   Building services...
[Build]   main Preparing...
[Info]    Building for armv7hf/raspberrypi3
[Info]    Emulation is enabled
[Debug]   Found build tasks:
[Debug]       main: build [.]
[Debug]   Resolving services with [raspberrypi3|armv7hf]
[Debug]   Found project types:
[Debug]       main: Dockerfile.template
[Debug]   Prepared tasks; building...
[Build]   main Step 1/5 : FROM balenalib/raspberrypi3-alpine
[Build]   main  ---> 588abea67f1d
[Build]   main Step 2/5 : COPY [".balena/qemu-execve","/tmp/qemu-execve"]
[Build]   main  ---> Using cache
[Build]   main  ---> 0d7a58a27580
[Build]   main Step 3/5 : RUN ["/tmp/qemu-execve","-execve","/bin/sh","-c","echo \"HELLO WORLD\" | cat"]
[Build]   main  ---> Using cache
[Build]   main  ---> f9765c5ad30d
[Build]   main Step 4/5 : ENV PATH="\"$PATH\""
[Build]   main  ---> Using cache
[Build]   main  ---> 8efbd64c6ab1
[Build]   main Step 5/5 : RUN ["/tmp/qemu-execve","-execve","/bin/sh","-c","echo \"HELLO WORLD\" | cat"]
[Build]   main  ---> [Warning] The requested image's platform (linux/arm/v7) does not match the detected host platform (linux/amd64) and no specific platform was requested
[Build]   main  ---> Running in 7f6435f621b5
[Build]   main /bin/sh: cat: not found
[Build]   Built 1 service in 0:05
[Error]   Build failed.
The command '/tmp/qemu-execve -execve /bin/sh -c echo "HELLO WORLD" | cat' returned a non-zero code: 127

Error: The command '/tmp/qemu-execve -execve /bin/sh -c echo "HELLO WORLD" | cat' returned a non-zero code: 127
    at Stream.<anonymous> (/usr/local/lib/balena-cli/node_modules/resin-docker-build/build/builder.js:181:23)
    at Stream.stream.write (/usr/local/lib/balena-cli/node_modules/through/index.js:26:11)
    at Stream.ondata (internal/streams/legacy.js:19:31)
    at Stream.emit (events.js:314:20)
    at Stream.EventEmitter.emit (domain.js:483:12)
    at drain (/usr/local/lib/balena-cli/node_modules/through/index.js:36:16)
    at Stream.stream.queue.stream.push (/usr/local/lib/balena-cli/node_modules/through/index.js:45:5)
    at Parser.parser.onToken (/usr/local/lib/balena-cli/node_modules/JSONStream/index.js:132:18)
    at Parser.proto.write (/usr/local/lib/balena-cli/node_modules/jsonparse/jsonparse.js:135:34)
    at Stream.<anonymous> (/usr/local/lib/balena-cli/node_modules/JSONStream/index.js:23:12)
    at Stream.stream.write (/usr/local/lib/balena-cli/node_modules/through/index.js:26:11)
    at IncomingMessage.ondata (_stream_readable.js:718:22)
    at IncomingMessage.emit (events.js:314:20)
    at IncomingMessage.EventEmitter.emit (domain.js:483:12)
    at addChunk (_stream_readable.js:297:12)
    at readableAddChunk (_stream_readable.js:272:9)
From previous event:
    at /usr/local/lib/balena-cli/node_modules/resin-docker-build/build/builder.js:73:20
    at processImmediate (internal/timers.js:461:21)
From previous event:
    at Builder.createBuildStream (/usr/local/lib/balena-cli/node_modules/resin-docker-build/build/builder.js:72:97)
    at /usr/local/lib/balena-cli/node_modules/resin-multibuild/build/build.js:89:21
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

For further help or support, visit:
https://www.balena.io/docs/reference/balena-cli/#support-faq-and-troubleshooting

Specifications

balena CLI version:

balena version -a
balena-cli version "13.3.1"
Node.js version "12.20.0"

Cloud backend: openBalena or balenaCloud? local build
Operating system version: e.g. Ubuntu 18.04
32/64 bit OS and processor: 64bit
Install method: executable installer

pipex commented 2 years ago

Get JF to create the link automatically, leaving it here for posterity https://jel.ly.fish/support-thread-1-0-0-front-cnv-ccbdaa5

jellyfish-bot commented 2 years ago

[alanb128] This issue has attached support thread https://jel.ly.fish/361c986a-6f1d-4110-ba4e-1b38295cb763

thgreasi commented 2 years ago

@pipex I think that this works for me with the latest version atm. Can you give it an extra try and if it works for you as well we can close this.

pipex commented 2 years ago

Sure, let me check if I still have the testing code though

pipex commented 2 years ago

Oh, nevermind, I forgot about the reproduction steps above

pipex commented 2 years ago

Yep, seems to be working for me now. There is no more qemu-execve though, did we change the way emulation works?

$ balena build --debug --deviceType raspberrypi3 --arch armv7hf --emulated
[debug] new argv=[/usr/local/lib/balena-cli/bin/node,/usr/local/lib/balena-cli/bin/run,build,--deviceType,raspberrypi3,--arch,armv7hf,--emulated] length=8
[debug] Deprecation check: 6.83001 days since last npm registry query for next major version release date.
[debug] Will not query the registry again until at least 7 days have passed.
[Debug]   Parsing input...
[Debug]   Loading project...
[Debug]   Resolving project...
[Info]    No "docker-compose.yml" file found at "/Users/flalanne/Development/balena/support/with-path"
[Info]    Creating default composition with source: "/Users/flalanne/Development/balena/support/with-path"
[Debug]   Creating project...
[Build]   Building services...
[Build]   main Preparing...
[Info]    Building for armv7hf/raspberrypi3
[Info]    Docker Desktop detected (daemon architecture: "x86_64")
[Info]      Docker itself will determine and enable architecture emulation if required,
[Info]      without balena-cli intervention and regardless of the --emulated option.
[Debug]   Found build tasks:
[Debug]       main: build [.]
[Debug]   Resolving services with [raspberrypi3|armv7hf]
[Debug]   Found project types:
[Debug]       main: Dockerfile.template
[Debug]   Prepared tasks; building...
[Build]   main Step 1/4 : FROM balenalib/raspberrypi3-alpine
[Build]   main  ---> 3d3155b4337a
[Build]   main Step 2/4 : RUN echo "HELLO WORLD" | cat
[Build]   main  ---> [Warning] The requested image's platform (linux/arm/v7) does not match the detected host platform (linux/amd64) and no specific platform was requested
[Build]   main  ---> Running in 9283ce9a82eb
[Build]   main HELLO WORLD
[Build]   main Removing intermediate container 9283ce9a82eb
[Build]   main  ---> 057feaa0e077
[Build]   main Step 3/4 : ENV PATH="$PATH"
[Build]   main  ---> [Warning] The requested image's platform (linux/arm/v7) does not match the detected host platform (linux/amd64) and no specific platform was requested
[Build]   main  ---> Running in 69197065b118
[Build]   main Removing intermediate container 69197065b118
[Build]   main  ---> e9d1f6f88acc
[Build]   main Step 4/4 : RUN echo "HELLO WORLD" | cat
[Build]   main  ---> [Warning] The requested image's platform (linux/arm/v7) does not match the detected host platform (linux/amd64) and no specific platform was requested
[Build]   main  ---> Running in a6cbb85f1bce
[Build]   main HELLO WORLD
[Build]   main Removing intermediate container a6cbb85f1bce
[Build]   main  ---> a40f882538e9
[Build]   main Successfully built a40f882538e9
[Build]   main Successfully tagged with-path_main:latest
[Build]   main Image size: 53.78 MB
[Build]   Built 1 service in 0:21
[Success] Build succeeded!

thgreasi commented 2 years ago

On a second try, ends up that it still fails for me @pipex , I guess I did something wrong initially. In my test qemu was used. Any chance you are now on an arm device?

pipex commented 2 years ago

No, i tested with my x86 machine @thgreasi, weird, did I mess up the replication? 😕

balena-io / balena-cli

Setting PATH in Dockerfile.template causes emulated build to fail #2475

Expected Behavior

Actual Behavior

Steps to Reproduce the Problem

Specifications