[Regression] Templates fail to work on Raspberry Pi with docker build

[alexellis]

The problem is that after adding buildx multi-arch support docker build and faas-cli build no longer work without intervention/additional flags when building on a non-Intel machine like RPi or ARM64.

Normal building on PCs/Intel is unaffected.

Expected Behaviour

The following should work on RPi without needing buildx, buildkit, or any build flags.

export DOCKER_CLI_EXPERIMENTAL=disabled
export DOCKER_BUILDKIT=0

faas-cli new --lang node12 test
faas-cli build -f test.yml

To be clear - faas-cli build just constructs a simple docker build statement.

The following can reproduce the error with docker build:

FROM --platform=${TARGETPLATFORM:-linux/amd64} node:12-alpine as ship
RUN ls

This fails on Raspberry Pi with docker build due to pulling the wrong base image, the one for linux/amd64.

Current Behaviour

It doesn't since we added headers to each Dockerfile for the TARGETPLATFORM such as:

FROM --platform=${TARGETPLATFORM:-linux/amd64} golang:1.13 as builder

However, If using buildx, it will work as expected because buildx inputs TARGETPLATFORM and BUILDPLATFORM

faas-cli publish -f test.yml

Possible Solution

When reproducing the issue, you can get a false positive, if you already have the correct base image in your library or Buildkit cache.

Always clear down images before attempting a repro:

docker images -q|xargs docker image rm -f
docker builder prune --all

Users can run faas-cli publish --platforms linux/arm/v7 on their desktop computers, then faas-cli deploy --gateway http://raspberry-pi-ip:8080

The platform flag can be overridden with plain Docker, but only if we move the ARG definition above the FROM line, so:

ARG TARGETPLATFORM
FROM --platform=${TARGETPLATFORM:-linux/amd64} golang:1.13 as builder

This initial change seems to have no knock-on effect and should be done quickly.

Unfortunately TARGETPLATFORM cannot be set to empty, so on the members call after we suggested that after the Dockerfiles are changed we exec docker version just before doing a build to inject the value for TARGETPLATFORM. Some remapping is required from linux/arm to a platform value like linux/arm/7.

Longer term I wonder if we can simply remove the line, and have docker build and buildx continue to work through defaults? So that would mean trying FROM golang:1.13 as builder

Context

Users cannot build with the OpenFaaS templates on ARM or ARMHF, unless they specifically pull or checkout an older version of the code such as 624efe5851de9e920fe986a079cbf8b24cc5a4cd

git clone https://github.com/openfaas/templates.git
cd templates
git checkout 624efe5851de9e920fe986a079cbf8b24cc5a4cd
cd ..
mv ./templates/template ./

faas-cli new --lang node12 test

[alexellis]

[fergalmoran]

Hi Alex - I still can't get it to build even with your instructions above?

➜  ~ lsb_release -a                                                                                                                                    [31/31]
No LSB modules are available.                                                                                                                                 
Distributor ID: Raspbian                                                                                                                                      
Description:    Raspbian GNU/Linux 10 (buster)                                                                                                                
Release:        10                                                                                                                                            
Codename:       buster                                                                                                                                        
➜  ~ docker --version                                                                                                                                         
Docker version 20.10.0, build 7287ab3                                                                                                                         
➜  ~ cd faas                                                                                                                                                  
➜  faas rm -rf *                                                                                                                                              
zsh: sure you want to delete all 4 files in /home/pi/faas [yn]? y                                                                                             
➜  faas faas-cli template pull                                                                                                                                
Fetch templates from repository: https://github.com/openfaas/templates.git at                                                                                 
2021/01/07 17:36:05 Attempting to expand templates from https://github.com/openfaas/templates.git                                                             
2021/01/07 17:36:10 Fetched 12 template(s) : [csharp dockerfile go java11 java11-vert-x node node12 php7 python python3 python3-debian ruby] from https://gith
ub.com/openfaas/templates.git                                                                                                                                 
➜  faas faas-cli new --lang node12 test                                                                                                                       
Folder: test created.                                                                                                                                         
  ___                   _____           ____                                                                                                                  
 / _ \ _ __   ___ _ __ |  ___|_ _  __ _/ ___|                                                                                                                 
| | | | '_ \ / _ \ '_ \| |_ / _` |/ _` \___ \                                                                                                                 
| |_| | |_) |  __/ | | |  _| (_| | (_| |___) |                                                                                                                
 \___/| .__/ \___|_| |_|_|  \__,_|\__,_|____/                                                                                                                 
      |_|                                                                                                                                                     

Function created in folder: test                                                                                                                              
Stack file written: test.yml                                                                                                                                  

Notes:                                                                                                                                                        
You have created a new function which uses Node.js 12 (TLS) and the OpenFaaS                                                                                  
of-watchdog which gives greater control over HTTP responses.                                                                                                  

npm i --save can be used to add third-party packages like request or cheerio
npm documentation: https://docs.npmjs.com/

Unit tests are run at build time via "npm run", edit package.json to specify 
how you want to execute them.

➜  faas faas-cli build -f test.yml     
[0] > Building test.
Clearing temporary build folder: ./build/test/
Preparing: ./test/ build/test/function
Building: test:latest with node12 template. Please wait..
Sending build context to Docker daemon  12.29kB
Step 1/31 : FROM --platform=${TARGETPLATFORM:-linux/amd64} openfaas/of-watchdog:0.7.2 as watchdog
 ---> bea76c093965
Step 2/31 : FROM --platform=${TARGETPLATFORM:-linux/amd64} node:12-alpine as ship
 ---> 0206ff8a5f9e
Step 3/31 : ARG TARGETPLATFORM
 ---> Using cache
 ---> 4b09d4b7adad
Step 4/31 : ARG BUILDPLATFORM
 ---> Using cache
 ---> beb3b5afa4ed
Step 5/31 : COPY --from=watchdog /fwatchdog /usr/bin/fwatchdog
 ---> Using cache
 ---> 8e5921f1c96d
Step 6/31 : RUN chmod +x /usr/bin/fwatchdog
 ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
 ---> Running in 6a6ba755faf0
standard_init_linux.go:219: exec user process caused: exec format error
The command '/bin/sh -c chmod +x /usr/bin/fwatchdog' returned a non-zero code: 1
[0] < Building test done in 3.30s.
[0] Worker done.

Total build time: 3.30s
Errors received during build:
- [test] received non-zero exit code from build, error: The command '/bin/sh -c chmod +x /usr/bin/fwatchdog' returned a non-zero code:
- 
- ```

[alexellis]

Hi, for Raspberry Pi you need to use the publish command, followed by deploy hope that helps. You may also need to delete those base images you downloaded.

I tend to build my RPi images on a Linux box or my Mac with the publish command now.

Happy to chat more on Slack if you want.

Alex

[fergalmoran]

That would be great, Alex. I'm a bit confused... I've pinged you an invite.

[alexellis]

So you'd just do the following from your PC, not on the RPi:

faas-cli publish --platforms linux/arm/v7
faas-cli deploy --gateway http://192.168.0.1:8080/

[alexellis]

/add label: wontfix

[alexellis]

The publish command means that we don't need to fix this issue. cc @LucasRoesler @utsavanand2 @viveksyngh @Waterdrips