Add new --remote-builder flag for remote builds via the Function Builder API

NikhilSharmaWe commented 1 year ago

Description

This PR works on adding the new remote-builder flag for faas-cli to be able to perform build operations from Function Builder API in Openfaas Pro. A new payload-secret flag is also added, in which the path of the payload.txt file containing the payload secret is specified.

Demo's :

Basic command : https://asciinema.org/a/LKpTzW0FhfoqHqbJKmIzkmcK1
Building 6 functions, 2 in parallel : https://asciinema.org/a/DZYpO2oQ7FbNvjoUCaTw3vtb1
Up command with remote-builder : https://asciinema.org/a/No1XmpjK6Fr7fXkYaBV4z1tNn
With Build Args : https://asciinema.org/a/vJPPSqm3GytD51M7wOC3lJvOT

Motivation and Context

[ ] I have raised an issue to propose this change (required) Fixes: https://github.com/openfaas/faas-cli/issues/931
How Has This Been Tested?

publish command with --remote-builder flag is successfully run


> faas-cli publish --remote-builder http://127.0.0.1:8081/build \
-f homepage.yml \
--payload-secret $HOME/.openfaas/payload.txt

Created buildx node: "multiarch" [0] > Building homepage. Clearing temporary build folder: ./build/homepage/ Preparing: ./homepage/ build/homepage/function Building: docker.io/nikhilsharmawe/homepage:latest with golang-http template. Please wait.. 1fa0f5174095a059ac57915d6838b0ec41d4223f154bfcdd9cd8068890dc507b 2023/05/10 17:20:41 homepage invoking the API for build at http://127.0.0.1:8081/build v: 2023-05-10T17:20:41Z [internal] load .dockerignore v: 2023-05-10T17:20:41Z [internal] load build definition from Dockerfile v: 2023-05-10T17:20:41Z [internal] load build definition from Dockerfile 0.00s v: 2023-05-10T17:20:41Z [internal] load .dockerignore 0.00s v: 2023-05-10T17:20:41Z [internal] load .dockerignore v: 2023-05-10T17:20:41Z [internal] load build definition from Dockerfile s: 2023-05-10T17:20:41Z transferring context: 0 s: 2023-05-10T17:20:41Z transferring context: 32 s: 2023-05-10T17:20:41Z transferring context: 41 s: 2023-05-10T17:20:41Z transferring dockerfile: 0 s: 2023-05-10T17:20:41Z transferring dockerfile: 30 s: 2023-05-10T17:20:41Z transferring dockerfile: 1821 v: 2023-05-10T17:20:41Z [internal] load .dockerignore 0.03s v: 2023-05-10T17:20:41Z [internal] load build definition from Dockerfile 0.05s v: 2023-05-10T17:20:41Z [internal] load metadata for docker.io/library/alpine:3.17.2 v: 2023-05-10T17:20:41Z [internal] load metadata for ghcr.io/openfaas/of-watchdog:0.9.11 v: 2023-05-10T17:20:41Z [internal] load metadata for docker.io/library/golang:1.19-alpine v: 2023-05-10T17:20:41Z [internal] load metadata for ghcr.io/openfaas/of-watchdog:0.9.11 0.55s v: 2023-05-10T17:20:41Z [internal] load metadata for docker.io/library/alpine:3.17.2 2.41s v: 2023-05-10T17:20:41Z [internal] load metadata for docker.io/library/golang:1.19-alpine 2.66s v: 2023-05-10T17:20:44Z [ship 1/7] FROM docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517 v: 2023-05-10T17:20:44Z [ship 2/7] RUN apk --no-cache add ca-certificates && addgroup -S app && adduser -S -g app app v: 2023-05-10T17:20:44Z [ship 3/7] RUN mkdir -p /home/app && chown app /home/app v: 2023-05-10T17:20:44Z [ship 4/7] WORKDIR /home/app v: 2023-05-10T17:20:44Z [build 1/14] FROM docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103 v: 2023-05-10T17:20:44Z [build 2/14] RUN apk --no-cache add git v: 2023-05-10T17:20:44Z [watchdog 1/1] FROM ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4 v: 2023-05-10T17:20:44Z [build 3/14] COPY --from=watchdog /fwatchdog /usr/bin/fwatchdog v: 2023-05-10T17:20:44Z [build 4/14] RUN chmod +x /usr/bin/fwatchdog v: 2023-05-10T17:20:44Z [build 5/14] RUN mkdir -p /go/src/handler v: 2023-05-10T17:20:44Z [build 6/14] WORKDIR /go/src/handler v: 2023-05-10T17:20:44Z [internal] load build context v: 2023-05-10T17:20:44Z [build 7/14] COPY . . v: 2023-05-10T17:20:44Z [build 8/14] RUN test -z "$(gofmt -l $(find . -type f -name '.go' -not -path "./vendor/" -not -path "./function/vendor/"))" || { echo "Run "gofmt -s -w" on your Golang code"; exit 1; } v: 2023-05-10T17:20:44Z [build 9/14] RUN sh ./modules-cleanup.sh v: 2023-05-10T17:20:44Z [build 10/14] WORKDIR /go/src/handler/function v: 2023-05-10T17:20:44Z [build 11/14] RUN mkdir -p /go/src/handler/function/static v: 2023-05-10T17:20:44Z [build 12/14] RUN GOOS=linux GOARCH=amd64 GOFLAGS= go test ./... -cover v: 2023-05-10T17:20:44Z [build 13/14] WORKDIR /go/src/handler v: 2023-05-10T17:20:44Z [build 14/14] RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GOFLAGS= go build --ldflags "-s -w" -o handler . v: 2023-05-10T17:20:44Z [ship 5/7] COPY --from=build --chown=app /go/src/handler/handler . v: 2023-05-10T17:20:44Z [ship 6/7] COPY --from=build --chown=app /usr/bin/fwatchdog . v: 2023-05-10T17:20:44Z [ship 7/7] COPY --from=build --chown=app /go/src/handler/function/static static v: 2023-05-10T17:20:44Z [build 1/14] FROM docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103 v: 2023-05-10T17:20:44Z [internal] load build context 0.00s v: 2023-05-10T17:20:44Z [ship 1/7] FROM docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517 v: 2023-05-10T17:20:44Z [watchdog 1/1] FROM ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4 s: 2023-05-10T17:20:44Z resolve docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103 0 s: 2023-05-10T17:20:44Z resolve docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517 0 s: 2023-05-10T17:20:44Z resolve ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4 0 s: 2023-05-10T17:20:44Z resolve ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4 0 v: 2023-05-10T17:20:44Z [watchdog 1/1] FROM ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4 0.11s s: 2023-05-10T17:20:44Z resolve docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103 0 v: 2023-05-10T17:20:44Z [build 1/14] FROM docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103 0.12s v: 2023-05-10T17:20:44Z [internal] load build context v: 2023-05-10T17:20:44Z [ship 1/7] FROM docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517 0.13s s: 2023-05-10T17:20:44Z resolve docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517 0 s: 2023-05-10T17:20:44Z transferring context: 0 s: 2023-05-10T17:20:44Z transferring context: 32 s: 2023-05-10T17:20:44Z transferring context: 12810 v: 2023-05-10T17:20:44Z [internal] load build context 0.02s v: 2023-05-10T17:20:44Z [ship 2/7] RUN apk --no-cache add ca-certificates && addgroup -S app && adduser -S -g app app 0.00s v: 2023-05-10T17:20:44Z [ship 3/7] RUN mkdir -p /home/app && chown app /home/app 0.00s v: 2023-05-10T17:20:44Z [ship 4/7] WORKDIR /home/app 0.00s v: 2023-05-10T17:20:44Z [build 2/14] RUN apk --no-cache add git 0.00s v: 2023-05-10T17:20:44Z [build 3/14] COPY --from=watchdog /fwatchdog /usr/bin/fwatchdog 0.00s v: 2023-05-10T17:20:44Z [build 4/14] RUN chmod +x /usr/bin/fwatchdog 0.00s v: 2023-05-10T17:20:44Z [build 5/14] RUN mkdir -p /go/src/handler 0.00s v: 2023-05-10T17:20:44Z [build 6/14] WORKDIR /go/src/handler 0.00s v: 2023-05-10T17:20:44Z [build 7/14] COPY . . 0.00s v: 2023-05-10T17:20:44Z [build 8/14] RUN test -z "$(gofmt -l $(find . -type f -name '.go' -not -path "./vendor/" -not -path "./function/vendor/"))" || { echo "Run "gofmt -s -w" on your Golang code"; exit 1; } 0.00s v: 2023-05-10T17:20:44Z [build 9/14] RUN sh ./modules-cleanup.sh 0.00s v: 2023-05-10T17:20:44Z [build 10/14] WORKDIR /go/src/handler/function 0.00s v: 2023-05-10T17:20:44Z [build 11/14] RUN mkdir -p /go/src/handler/function/static 0.00s v: 2023-05-10T17:20:44Z [build 12/14] RUN GOOS=linux GOARCH=amd64 GOFLAGS= go test ./... -cover 0.00s v: 2023-05-10T17:20:44Z [build 13/14] WORKDIR /go/src/handler 0.00s v: 2023-05-10T17:20:44Z [build 14/14] RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GOFLAGS= go build --ldflags "-s -w" -o handler . 0.00s v: 2023-05-10T17:20:44Z [ship 5/7] COPY --from=build --chown=app /go/src/handler/handler . 0.00s v: 2023-05-10T17:20:44Z [ship 6/7] COPY --from=build --chown=app /usr/bin/fwatchdog . 0.00s v: 2023-05-10T17:20:44Z [ship 7/7] COPY --from=build --chown=app /go/src/handler/function/static static 0.00s v: 2023-05-10T17:20:44Z exporting to image s: 2023-05-10T17:20:44Z exporting layers 0 s: 2023-05-10T17:20:44Z exporting layers 0 s: 2023-05-10T17:20:44Z exporting manifest sha256:bdd4df0c1bab88fad637972342d4d946bcdab79892fb1d7dc5efc6870ad22eb8 0 s: 2023-05-10T17:20:44Z exporting manifest sha256:bdd4df0c1bab88fad637972342d4d946bcdab79892fb1d7dc5efc6870ad22eb8 0 s: 2023-05-10T17:20:44Z exporting config sha256:af35980857a8bc06d203f69883ffa90bb4114871f73a76818e0f0b2c116db6b0 0 s: 2023-05-10T17:20:44Z exporting config sha256:af35980857a8bc06d203f69883ffa90bb4114871f73a76818e0f0b2c116db6b0 0 s: 2023-05-10T17:20:44Z pushing layers 0 s: 2023-05-10T17:20:46Z pushing layers 0 s: 2023-05-10T17:20:46Z pushing manifest for docker.io/nikhilsharmawe/homepage:latest@sha256:bdd4df0c1bab88fad637972342d4d946bcdab79892fb1d7dc5efc6870ad22eb8 0 s: 2023-05-10T17:20:46Z pushing manifest for docker.io/nikhilsharmawe/homepage:latest@sha256:bdd4df0c1bab88fad637972342d4d946bcdab79892fb1d7dc5efc6870ad22eb8 0 v: 2023-05-10T17:20:44Z exporting to image 1.78s 2023/05/10 17:20:46 homepage success building and pushing image: docker.io/nikhilsharmawe/homepage:latest [0] < Building homepage done in 4.75s. [0] Worker done.

Total build time: 4.75s


- and the created image is successfully deployed:

faas-cli deploy --image nikhilsharmawe/homepage:latest

Deployed. 202 Accepted. URL: http://127.0.0.1:8080/function/homepage

- in case of a syntax error in the handler func:

faas-cli publish --remote-builder http://127.0.0.1:8081/build \ -f homepage.yml \ --payload-secret $HOME/.openfaas/payload.txt

Created buildx node: "multiarch" [0] > Building homepage. Clearing temporary build folder: ./build/homepage/ Preparing: ./homepage/ build/homepage/function Building: docker.io/nikhilsharmawe/homepage:latest with golang-http template. Please wait.. 2b2a1a5b83e81c1a724ccf80a8871086b595ddba08ed24e63669288f056b6df0 2023/05/10 17:23:29 homepage invoking the API for build at http://127.0.0.1:8081/build v: 2023-05-10T17:23:29Z [internal] load .dockerignore v: 2023-05-10T17:23:29Z [internal] load build definition from Dockerfile v: 2023-05-10T17:23:29Z [internal] load .dockerignore 0.00s v: 2023-05-10T17:23:29Z [internal] load build definition from Dockerfile 0.00s v: 2023-05-10T17:23:29Z [internal] load build definition from Dockerfile v: 2023-05-10T17:23:29Z [internal] load .dockerignore s: 2023-05-10T17:23:29Z transferring dockerfile: 0 s: 2023-05-10T17:23:29Z transferring dockerfile: 30 s: 2023-05-10T17:23:29Z transferring dockerfile: 1821 s: 2023-05-10T17:23:29Z transferring context: 0 s: 2023-05-10T17:23:29Z transferring context: 32 s: 2023-05-10T17:23:29Z transferring context: 41 v: 2023-05-10T17:23:29Z [internal] load build definition from Dockerfile 0.11s v: 2023-05-10T17:23:29Z [internal] load .dockerignore 0.17s v: 2023-05-10T17:23:29Z [internal] load metadata for docker.io/library/alpine:3.17.2 v: 2023-05-10T17:23:29Z [internal] load metadata for ghcr.io/openfaas/of-watchdog:0.9.11 v: 2023-05-10T17:23:29Z [internal] load metadata for docker.io/library/golang:1.19-alpine v: 2023-05-10T17:23:29Z [internal] load metadata for ghcr.io/openfaas/of-watchdog:0.9.11 0.38s v: 2023-05-10T17:23:29Z [internal] load metadata for docker.io/library/alpine:3.17.2 2.24s v: 2023-05-10T17:23:29Z [internal] load metadata for docker.io/library/golang:1.19-alpine 2.29s v: 2023-05-10T17:23:31Z [ship 1/7] FROM docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517 v: 2023-05-10T17:23:31Z [ship 2/7] RUN apk --no-cache add ca-certificates && addgroup -S app && adduser -S -g app app v: 2023-05-10T17:23:31Z [ship 3/7] RUN mkdir -p /home/app && chown app /home/app v: 2023-05-10T17:23:31Z [ship 4/7] WORKDIR /home/app v: 2023-05-10T17:23:31Z [build 1/14] FROM docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103 v: 2023-05-10T17:23:31Z [build 2/14] RUN apk --no-cache add git v: 2023-05-10T17:23:31Z [watchdog 1/1] FROM ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4 v: 2023-05-10T17:23:31Z [build 3/14] COPY --from=watchdog /fwatchdog /usr/bin/fwatchdog v: 2023-05-10T17:23:31Z [build 4/14] RUN chmod +x /usr/bin/fwatchdog v: 2023-05-10T17:23:31Z [build 5/14] RUN mkdir -p /go/src/handler v: 2023-05-10T17:23:31Z [build 6/14] WORKDIR /go/src/handler v: 2023-05-10T17:23:31Z [internal] load build context v: 2023-05-10T17:23:31Z [build 7/14] COPY . . v: 2023-05-10T17:23:31Z [build 8/14] RUN test -z "$(gofmt -l $(find . -type f -name '.go' -not -path "./vendor/" -not -path "./function/vendor/"))" || { echo "Run "gofmt -s -w" on your Golang code"; exit 1; } v: 2023-05-10T17:23:31Z [build 9/14] RUN sh ./modules-cleanup.sh v: 2023-05-10T17:23:31Z [build 10/14] WORKDIR /go/src/handler/function v: 2023-05-10T17:23:31Z [build 11/14] RUN mkdir -p /go/src/handler/function/static v: 2023-05-10T17:23:31Z [build 12/14] RUN GOOS=linux GOARCH=amd64 GOFLAGS= go test ./... -cover v: 2023-05-10T17:23:31Z [build 13/14] WORKDIR /go/src/handler v: 2023-05-10T17:23:31Z [build 14/14] RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GOFLAGS= go build --ldflags "-s -w" -o handler . v: 2023-05-10T17:23:31Z [ship 5/7] COPY --from=build --chown=app /go/src/handler/handler . v: 2023-05-10T17:23:31Z [ship 6/7] COPY --from=build --chown=app /usr/bin/fwatchdog . v: 2023-05-10T17:23:31Z [ship 7/7] COPY --from=build --chown=app /go/src/handler/function/static static v: 2023-05-10T17:23:31Z [build 1/14] FROM docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103 v: 2023-05-10T17:23:31Z [internal] load build context 0.00s v: 2023-05-10T17:23:31Z [watchdog 1/1] FROM ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4 v: 2023-05-10T17:23:31Z [ship 1/7] FROM docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517 s: 2023-05-10T17:23:31Z resolve docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103 0 s: 2023-05-10T17:23:32Z resolve ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4 0 s: 2023-05-10T17:23:32Z resolve docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517 0 v: 2023-05-10T17:23:31Z [build 1/14] FROM docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103 0.22s s: 2023-05-10T17:23:32Z resolve docker.io/library/golang:1.19-alpine@sha256:9668643a2e62d8bd298ef3663a96de4a70ceb2865b9b7cadd1d5e08387745103 0 s: 2023-05-10T17:23:32Z resolve ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4 0 v: 2023-05-10T17:23:31Z [watchdog 1/1] FROM ghcr.io/openfaas/of-watchdog:0.9.11@sha256:b84fd6db48e31e3d65dcd36672bf49b7a7b3dfced49ce602f51e877d9d1d56e4 0.25s v: 2023-05-10T17:23:32Z [internal] load build context v: 2023-05-10T17:23:31Z [ship 1/7] FROM docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517 0.26s s: 2023-05-10T17:23:32Z resolve docker.io/library/alpine:3.17.2@sha256:ff6bdca1701f3a8a67e328815ff2346b0e4067d32ec36b7992c1fdc001dc8517 0 s: 2023-05-10T17:23:32Z transferring context: 0 s: 2023-05-10T17:23:32Z transferring context: 32 s: 2023-05-10T17:23:32Z transferring context: 12809 v: 2023-05-10T17:23:32Z [internal] load build context 0.05s v: 2023-05-10T17:23:32Z [build 2/14] RUN apk --no-cache add git 0.00s v: 2023-05-10T17:23:32Z [build 3/14] COPY --from=watchdog /fwatchdog /usr/bin/fwatchdog 0.00s v: 2023-05-10T17:23:32Z [build 4/14] RUN chmod +x /usr/bin/fwatchdog 0.00s v: 2023-05-10T17:23:32Z [build 5/14] RUN mkdir -p /go/src/handler 0.00s v: 2023-05-10T17:23:32Z [build 6/14] WORKDIR /go/src/handler 0.00s v: 2023-05-10T17:23:32Z [build 7/14] COPY . . 0.00s v: 2023-05-10T17:23:32Z [build 8/14] RUN test -z "$(gofmt -l $(find . -type f -name '.go' -not -path "./vendor/" -not -path "./function/vendor/"))" || { echo "Run "gofmt -s -w" on your Golang code"; exit 1; } 0.00s v: 2023-05-10T17:23:32Z [build 9/14] RUN sh ./modules-cleanup.sh 0.00s v: 2023-05-10T17:23:32Z [build 10/14] WORKDIR /go/src/handler/function 0.00s v: 2023-05-10T17:23:32Z [build 11/14] RUN mkdir -p /go/src/handler/function/static 0.00s v: 2023-05-10T17:23:32Z [build 12/14] RUN GOOS=linux GOARCH=amd64 GOFLAGS= go test ./... -cover l: 2023-05-10T17:23:32Z go: downloading github.com/openfaas/templates-sdk/go-http v0.0.0-20220408082716-5981c545cb03

l: 2023-05-10T17:23:38Z # handler/function ./handler.go:15:3: syntax error: unexpected return, expecting expression

v: 2023-05-10T17:23:32Z [build 12/14] RUN GOOS=linux GOARCH=amd64 GOFLAGS= go test ./... -cover 7.64s 500 [0] < Building homepage done in 10.54s. [0] Worker done.

Total build time: 10.54s Errors received during build:

homepage failure while building or pushing image docker.io/nikhilsharmawe/homepage:latest: failure: failed to solve: process "/bin/sh -c GOOS=${TARGETOS} GOARCH=${TARGETARCH} GOFLAGS=${GOFLAGS} go test ./... -cover" did not complete successfully: exit code: 2

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I've read the CONTRIBUTION guide
[x] I have signed-off my commits with git commit -s
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.

alexellis commented 1 year ago

Thanks for the contribution.

Generally, we need to see "How Has This Been Tested?" fully populated before we can consider a review.

alexellis commented 1 year ago

How Has This Been Tested? Is still missing.

alexellis commented 1 year ago

is run successfully with logging message: Success building image

Please share the output and verify that faas-cli deploy can use the image.

I'd expect you to use a stack.yml file without giving any extra arguments - this all gets loaded by the YAML file

alexellis commented 1 year ago

The output looks a lot better. But I can't see the response from the server? There should be build logs.

We'd also want to do a build for a Go function with a syntax error so that we can prove the error flow works and gives the user the problem on the console whilst exiting non zero.

NikhilSharmaWe commented 1 year ago

@alexellis When we try to use go function with a syntax error in the handler code, this error is logged:

Created buildx node: "multiarch"
[0] > Building homepage.
Clearing temporary build folder: ./build/homepage/
Preparing: ./homepage/ build/homepage/function
Building: docker.io/nikhilsharmawe/homepage:latest with golang-http template. Please wait..
/tmp/builder-3367007414/req.tar
20bd31cea19229a440689707dd1c922317f27e8747772f88da69a5723c98bb00
500
2023/04/30 09:35:54 Unable to build image docker.io/nikhilsharmawe/homepage:latest: failure: failed to solve: process "/bin/sh -c GOOS=${TARGETOS} GOARCH=${TARGETARCH} GOFLAGS=${GOFLAGS} go test ./... -cover" did not complete successfully: exit code: 2

Is this what you expect?

or we expect detailed logs something like we get in the case of build, which also gives user info about the statement responsible for the issue and solution:

Step 18/37 : RUN test -z "$(gofmt -l $(find . -type f -name '*.go' -not -path "./vendor/*" -not -path "./function/vendor/*"))" || { echo "Run \"gofmt -s -w\" on your Golang code"; exit 1; }
 ---> Running in 22a6770aa843
./function/handler.go:15:3: expected operand, found 'return'

But since we are not doing docker build / buildx here in remote-builder, these will not be logged in that case.

Please give your thoughts.

alexellis commented 1 year ago

The API call that you're making should have both a status code to determine errors and a set of logs. Look at the docs for the format.

I also thought that authentication was required for the builder API - the payload secret?

NikhilSharmaWe commented 1 year ago

@alexellis The payload secret is fetched through:

kubectl get secret \
    -n openfaas payload-secret -o jsonpath='{.data.payload-secret}' \
    | base64 --decode \
    > payload.txt

I currently do it manually.

alexellis commented 1 year ago

I'm just wondering how you are able to only pass in a single argument here?

Shouldn't there be an additional argument for the payload secret?

alexellis commented 1 year ago

We discussed the approach on the weekly call, I think you may have dropped off by then.

faas-cli shouldn't be calling any exec commands to kubectl.

In this case, have the user pass in the password via a file.

I'm not sure that having it go in via a flag is a good idea as it will get leaked in bash history.

--payload-secret $HOME/.openfaas/payload-secret

Could you make the change and demo it to us on the community call in ~ 1.5 hours from now?

https://docs.openfaas.com/community/

alexellis commented 1 year ago

Could you try this also with faas-cli up --remote-builder etc?

alexellis commented 1 year ago

Hi, please could you rebase this PR?

Also run git commit --amend -s to set the title to: Add --remote-builder flag to publish command

In the commit body, you can write anything else you want to summarise - including wrapping the text to 80 chars.

This is what I strive to follow myself - https://cbea.ms/git-commit/

alexellis commented 1 year ago

@welteki I'd like you to try this out on your machine for a regular build/publish (unaffected) and for a remote build - with one single build and multiple - also trying out build-args.

Thanks

welteki commented 1 year ago

@welteki I'd like you to try this out on your machine for a regular build/publish (unaffected) and for a remote build - with one single build and multiple - also trying out build-args.

Tested this:

✅ build/publish without build args
✅ remote build/publish without build args
✅ build/publish with build args
❌ remote build/publish with build args

Build args are not working for remote builds.

I used the python-http template and tried setting a custom python version through build args.

faas-cli template store pull python3-http

faas-cli new python-fn --lang python3-http --prefix ttl.sh

faas-cli up -f python-fn.yml \
  --build-arg PYTHON_VERSION=3.10 \
  --remote-builder http://127.0.0.1:8081/build \
  --payload-secret ./payload.txt

The remote builder builds the function but in the logs I see lines like:

v: 2023-05-31T11:09:55Z [build  1/16] FROM docker.io/library/python:3.11-alpine@sha256:4e8e9a59bf1b3ca8e030244bc5f801f23e41e37971907371da21191312087a07

It uses the default python version 3.11 instead of the version set in the build arg. This indicates the build arg is not passed to the remote builder.

NikhilSharmaWe commented 1 year ago

@welteki I have found the fix. Currently, I am traveling but will fix it asap and will ping you for a review after.

Thank you for testing and informing the issue.

alexellis commented 1 year ago

@NikhilSharmaWe can you try the example provided by Han please?

NikhilSharmaWe commented 1 year ago

@alexellis I have tested and it works fine now with Build Args.

alexellis commented 1 year ago

@NikhilSharmaWe please could you add example commands and usage under the curl instructions in the docs?

Clone the repo and run it locally so you can test the layout etc.

https://github.com/openfaas/docs/blob/master/docs/openfaas-pro/builder.md

https://docs.openfaas.com/openfaas-pro/builder/

openfaas / faas-cli