Closed Dentrax closed 2 years ago
@LucasRoesler ptal
My initial thoughts are:
Turn go modules off (the build arg, but don't delete anything), and it should continue to work.
Or remove vendoring and it will continue to work.
Vendoring and modules together (in the build) have started to fail in recent Go versions.
Please try both and report back.
Hmm, now I just turn off GO111MODULE
flag (by setting off
) and removed it from the build flags. Vendored the function afterward: $ go mod vendor
Although $ go build --ldflags "-s -w" -a -installsuffix cgo -o handler .
works well manually in the function directory, I get another error during the Docker build phase:
$ faas-cli up -f notifier-fn.yml
# handler
./main.go:102:39: cannot use req (type "handler/vendor/github.com/openfaas/templates-sdk/go-http".Request) as type "handler/function/vendor/github.com/openfaas/templates-sdk/go-http".Request in argument to function.Handle
Setting the Go version to 1.15
did not resolve the issue. 🤔
@Dentrax i have noticed this issue as well. I ran into it last week, but I have not found a backwards compatible solution. Last weekend when I ran in to the issue, i opted to use the http middleware template instead as a quick fix. But I am still investigating how to fix this template. I need to determine if the middleware template is actually working or just lucky because it doesn't import any external packages.
@LucasRoesler Thanks for the detailed explanation. We have moved the function from golang-http
to golang-middleware
, as you said. Now it rocks. ^^
$ faas-cli new notifier2-fn --lang golang-middleware --prefix Dentrax
I think it was some kind of specific bug at the golang-http
template. Version: github.com/openfaas/templates-sdk@v0.0.0-20200723092016-0ebf61253625
@alexellis I started a looking at a fix that would simply aligned the http and the middleware templates. But there is an issue.
The http template includes a vendor folder, this is a problem in vendor module. It produces this error due to Go validating that the mod file and the vendor folder are in sync, this produces the following error
#25 0.437 go: inconsistent vendoring in /go/src/handler:
#25 0.437 github.com/openfaas/templates-sdk@v0.0.0-20200723092016-0ebf61253625: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
#25 0.437 handler/function: is replaced in go.mod, but not marked as replaced in vendor/modules.txt
#25 0.437
#25 0.437 run 'go mod vendor' to sync, or use -mod=mod or -mod=readonly to ignore the vendor directory
To work around this error, you can delete the vendor folder that is included in the template. When you remove the vendor folder, I have a PR that works in these two scenarios
faas-cli build -f publisher.yml --build-arg GO111MODULE=on
cd publisher && go mod vendor && cd ..
faas-cli build -f publisher.yml --build-arg GO111MODULE=off
But you can't use modules and vendor, for example this will fail
cd publisher && go mod vendor && cd ..
faas-cli build -f publisher.yml --build-arg GO111MODULE=on
This behavior is effectively implemented by using
RUN if [ -d ./function/vendor ] && [ "${GO111MODULE}" == "off" ]; then \
echo "moving handler vendor" && \
mv -f ./function/vendor .; \
else \
echo "using modules or vendor not found "; \
fi
the biggest issue is that the template currently defaults to modules "off", which means the default vendor folder that i just removed is required to make the default "echo" handler work. To make it work we would need to
One thing to note, i tried using go mod tidy
to synchronize the go.mod and vendor during the Docker build, but it still results in and error
#20 [build 8/14] RUN cat function/GO_REPLACE.txt >> ./go.mod || exit 0
#20 sha256:48ad39e8ada37838ac158b13cce56b5bb4cd5dc818c058f9c1f1bc40c024a10f
#20 0.308 cat: can't open 'function/GO_REPLACE.txt': No such file or directory
#20 DONE 0.3s
#21 [build 9/14] RUN if [ -d ./function/vendor ]; then echo "moving handler vendor" && mv -f ./function/vendor . && go mod tidy; else echo "using modules or vendor not found "; fi
#21 sha256:6300638d9169e81111fb8f382fd8a274d66ee6ec190eb958125fe17c0a0361ed
#21 0.357 moving handler vendor
#21 0.539 go: downloading github.com/openfaas/templates-sdk v0.0.0-20200723092016-0ebf61253625
#21 0.631 go: found handler/function in handler/function v0.0.0-00010101000000-000000000000
#21 2.024 go: downloading github.com/openfaas/templates-sdk v0.0.0-20200723110415-a699ec277c12
#21 2.037 go: downloading github.com/nats-io/nats.go v1.10.0
#21 2.140 go: downloading github.com/nats-io/jwt v1.1.0
#21 2.140 go: downloading github.com/nats-io/nkeys v0.1.4
#21 2.140 go: downloading github.com/nats-io/nuid v1.0.1
#21 2.141 go: downloading github.com/nats-io/nats-server/v2 v2.1.9
#21 2.141 go: downloading github.com/golang/protobuf v1.4.3
#21 2.141 go: downloading google.golang.org/protobuf v1.25.0
#21 2.182 go: downloading golang.org/x/crypto v0.0.0-20200323165209-0ec3e9974c59
#21 2.388 go: downloading github.com/google/go-cmp v0.5.0
#21 2.487 go: downloading golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e
#21 2.517 go: downloading golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543
#21 DONE 2.9s
#22 [build 10/14] RUN test -z "$(gofmt -l $(find . -type f -name '*.go' -not -path "./vendor/*" -not -path "./function/vendor/*"))" || { echo "Run "gofmt -s -w" on your Golang code"; exit 1; }
#22 sha256:310a7c685f0ed22d53cc736699d8299ff5543963680ef2eefbb5349f9c7dc1ac
#22 DONE 0.4s
#23 [build 11/14] WORKDIR /go/src/handler/function
#23 sha256:7c444425d527f8de94638a1c47c4fe729f0f40b6ca22c8c74f391558668fdded
#23 DONE 0.0s
#24 [build 12/14] RUN GOOS=linux GOARCH=amd64 go test ./... -cover
#24 sha256:0c7c7857b64200523e078c51cca979b4108c14c4694b7e51299949370e2f4c68
#24 4.240 ? function [no test files]
#24 DONE 4.3s
#25 [build 13/14] WORKDIR /go/src/handler
#25 sha256:f548c022b481872d9036a04d81a4cd4f559d77290190882ef0fd5bf90e13b517
#25 DONE 0.0s
#26 [build 14/14] RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build --ldflags "-s -w" -a -installsuffix cgo -o handler .
#26 sha256:63f06c1f4ebe4dd80dbd0f24d262bb9d0cb9f39f60acfc1fda7eb06bbbb9ad72
#26 0.492 go: inconsistent vendoring in /go/src/handler:
#26 0.492 handler/function@v0.0.0-00010101000000-000000000000: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
#26 0.492 handler/function: is replaced in go.mod, but not marked as replaced in vendor/modules.txt
#26 0.492 github.com/nats-io/nats.go@v1.10.0: is marked as explicit in vendor/modules.txt, but not explicitly required in go.mod
#26 0.492
#26 0.492 run 'go mod vendor' to sync, or use -mod=mod or -mod=readonly to ignore the vendor directory
#26 ERROR: executor failed running [/bin/sh -c CGO_ENABLED=${CGO_ENABLED} GOOS=${TARGETOS} GOARCH=${TARGETARCH} go build --ldflags "-s -w" -a -installsuffix cgo -o handler .]: exit code: 1
The cause is the replace we use in the template go.mod
Do you know if these steps affect caching? Can you try some things and find out.
Do you know if these steps affect caching? Can you try some things and find out.
@alexellis what are some things that you consider as proof? just running the build twice and seeing that step reported as cached?
Here is a second build of my test function, all of the steps report as cached
$ faas-cli build -f publisher.yml --build-arg GO111MODULE=on
[0] > Building publisher.
Clearing temporary build folder: ./build/publisher/
Preparing: ./publisher/ build/publisher/function
Building: publisher:latest with golang-http template. Please wait..
#1 [internal] load build definition from Dockerfile
#1 sha256:21a3073c4b533f5a50a7af91c6fc092fd2e018454c212c837e9a49b96a9ee5d5
#1 transferring dockerfile: 2.01kB done
#1 DONE 0.0s
#2 [internal] load .dockerignore
#2 sha256:10bc899bd437f5c8b9341111d3de92a13762288bed34430aa2ff9bafe5407c5a
#2 transferring context: 102B done
#2 DONE 0.0s
#5 [internal] load metadata for docker.io/library/golang:1.15-alpine3.13
#5 sha256:58dd66b3cb84deb634620790045bb91934c3fb27560259f0b5bb69b42224edb1
#5 ...
#4 [internal] load metadata for ghcr.io/openfaas/of-watchdog:0.8.4
#4 sha256:a36b81af230f4aa9af61173e476a504ddf679531cc6fb84c21a9f883d305ce12
#4 DONE 0.2s
#3 [internal] load metadata for docker.io/library/alpine:3.13
#3 sha256:5ecd52757bf8256597d3bd40d120a58086e4bc5517721ddf15a4b534c85228a0
#3 DONE 0.6s
#5 [internal] load metadata for docker.io/library/golang:1.15-alpine3.13
#5 sha256:58dd66b3cb84deb634620790045bb91934c3fb27560259f0b5bb69b42224edb1
#5 DONE 0.6s
#6 [stage-2 1/7] FROM docker.io/library/alpine:3.13@sha256:a75afd8b57e7f34e4dad8d65e2c7ba2e1975c795ce1ee22fa34f8cf46f96a3be
#6 sha256:ac117f80319f3dfc5f3a89bed8fde570a7095ef011b4aa4f1832954fe5fae646
#6 DONE 0.0s
#10 [build 1/14] FROM docker.io/library/golang:1.15-alpine3.13@sha256:a025015951720f3227acd51b0a99a71578b574a63172ea7d2415c60ae5e2bc0a
#10 sha256:945f28b33676aa8e1e61782ed1c14580cb6577c9dae69198435ffaa8642d6acc
#10 DONE 0.0s
#12 [watchdog 1/1] FROM ghcr.io/openfaas/of-watchdog:0.8.4@sha256:9a0b595f77724005ef7336d68a5a6347426d1931c82f238916496f27d7ea2e74
#12 sha256:0b83d079b8d19e155cadf89ef4c3782ffd5a248d98dbd78ee94c18245505455c
#12 DONE 0.0s
#17 [internal] load build context
#17 sha256:f377bbe6ad24eb025bf76e33d6219983965cc07c9dd78e02186bb94c2e8061ba
#17 transferring context: 16.92kB done
#17 DONE 0.0s
#15 [build 5/14] RUN mkdir -p /go/src/handler
#15 sha256:49827b6b29acd15ceb89996f6858464d41ef41d45bf4439caa09ad778a541c9b
#15 CACHED
#27 [stage-2 6/7] COPY --from=build --chown=app /usr/bin/fwatchdog .
#27 sha256:a15f3a00af062c11966cc8bf9b62eec7189a69a7a5d6bd292268f5c04e62e80f
#27 CACHED
#13 [build 3/14] COPY --from=watchdog /fwatchdog /usr/bin/fwatchdog
#13 sha256:3d4b56eeab2f493ee25bcffa0a77d52bf329e8d5bcf1a7f4f9e60d76207b024c
#13 CACHED
#16 [build 6/14] WORKDIR /go/src/handler
#16 sha256:2d57cb8be74105b3bf574dde9b3acfb2afb9ef7c3db0babaf2976e90bd189f0a
#16 CACHED
#9 [stage-2 4/7] WORKDIR /home/app
#9 sha256:8d07b104cc02b839e2f8b214eb64d1906c976d87c5178fc380b99ff4ee79f52d
#9 CACHED
#14 [build 4/14] RUN chmod +x /usr/bin/fwatchdog
#14 sha256:0ce54538eaf598232623e0d0131601ae53ed928d280203d11398984ad325296d
#14 CACHED
#26 [stage-2 5/7] COPY --from=build --chown=app /go/src/handler/handler .
#26 sha256:96883fc3eb65c24f97b0347d0c5bfb829b3bd29c2f2df68a7681a0257c0a09d3
#26 CACHED
#11 [build 2/14] RUN apk --no-cache add git
#11 sha256:e895da5279e30155cbfeb55e93d54399f92e5b71ef96937c7955731c811f752e
#11 CACHED
#25 [build 14/14] RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build --ldflags "-s -w" -a -installsuffix cgo -o handler .
#25 sha256:f103a02f229f25654f54cbde12afa99aa0c721c85804b51c23e539cb95e044e9
#25 CACHED
#21 [build 10/14] RUN test -z "$(gofmt -l $(find . -type f -name '*.go' -not -path "./vendor/*" -not -path "./function/vendor/*"))" || { echo "Run "gofmt -s -w" on your Golang code"; exit 1; }
#21 sha256:b1d524d09ab70ce20b7f57efa46db80d3cf0e49ea734b78f152828451eacc2cd
#21 CACHED
#23 [build 12/14] RUN GOOS=linux GOARCH=amd64 go test ./... -cover
#23 sha256:2d9eb2ff7aff1812f647f4c7f9e1b420821e818717ce1145553f6b44635b0073
#23 CACHED
#19 [build 8/14] RUN cat function/GO_REPLACE.txt >> ./go.mod || exit 0
#19 sha256:4d72ddb9b23a8417c2db4826354610584a497c771deef7debf52befd032d9638
#19 CACHED
#20 [build 9/14] RUN if [ -d ./function/vendor ]; then echo "moving handler vendor" && mv -f ./function/vendor . && go mod tidy; else echo "using modules or vendor not found "; fi
#20 sha256:d1efda4d0f61d94cc51ed7d079ca194f3ff8a5dae60a6d9af48ffe4bc320efb8
#20 CACHED
#22 [build 11/14] WORKDIR /go/src/handler/function
#22 sha256:10fbc5207db3831f3e06a80dc0c6f960ca19f7dbd8192ff2435b41aaa663a832
#22 CACHED
#24 [build 13/14] WORKDIR /go/src/handler
#24 sha256:ee155e1a79e01bab85a5c637cbf925c23746c9aedd5b1288b142e5628df20daf
#24 CACHED
#8 [stage-2 3/7] RUN mkdir -p /home/app && chown app /home/app
#8 sha256:392db714e3258ded26fd35e24dbf9da4b34d5f1b72bafa6651edb2c67c6e337a
#8 CACHED
#7 [stage-2 2/7] RUN apk --no-cache add ca-certificates && addgroup -S app && adduser -S -g app app
#7 sha256:f3920e39e6cb6756252ef8376a8a6a562e10f4156f6387b0e5aa5f3204bbbeb5
#7 CACHED
#18 [build 7/14] COPY . .
#18 sha256:b2081521828c5a6f4f2df32505d0dfa0fd8e3b7f26e1e4e23ec1cfbdc2e60fb5
#18 CACHED
#28 [stage-2 7/7] COPY --from=build --chown=app /go/src/handler/function/ .
#28 sha256:0f57dc249c06f1ea6f7b251e043aaa88a4977b712ae899c41a93513ed9e7578a
#28 CACHED
#29 exporting to image
#29 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#29 exporting layers done
#29 writing image sha256:ae862458b5fc8f845899815ba391511beb488b07239a5d79c2f284f60a351e9b done
#29 naming to docker.io/library/publisher:latest done
#29 DONE 0.0s
Image: publisher:latest built.
[0] < Building publisher done in 2.03s.
[0] Worker done.
Total build time: 2.03s
Same with modules off and a vendor folder exists
$ faas-cli build -f publisher.yml --build-arg GO111MODULE=off
[0] > Building publisher.
Clearing temporary build folder: ./build/publisher/
Preparing: ./publisher/ build/publisher/function
Building: publisher:latest with golang-http template. Please wait..
#1 [internal] load build definition from Dockerfile
#1 sha256:e574877995c10303eb346abe88f875984739ad086d155a9a1760ba721dc461b9
#1 transferring dockerfile: 1.99kB done
#1 DONE 0.0s
#2 [internal] load .dockerignore
#2 sha256:c2879ad2d201614c4ca5f0b0a93e9bd95ddcd2d3244b2afe07de6e6ac0eda3b2
#2 transferring context: 102B done
#2 DONE 0.0s
#5 [internal] load metadata for docker.io/library/golang:1.15-alpine3.13
#5 sha256:58dd66b3cb84deb634620790045bb91934c3fb27560259f0b5bb69b42224edb1
#5 ...
#4 [internal] load metadata for ghcr.io/openfaas/of-watchdog:0.8.4
#4 sha256:a36b81af230f4aa9af61173e476a504ddf679531cc6fb84c21a9f883d305ce12
#4 DONE 0.2s
#3 [internal] load metadata for docker.io/library/alpine:3.13
#3 sha256:5ecd52757bf8256597d3bd40d120a58086e4bc5517721ddf15a4b534c85228a0
#3 ...
#5 [internal] load metadata for docker.io/library/golang:1.15-alpine3.13
#5 sha256:58dd66b3cb84deb634620790045bb91934c3fb27560259f0b5bb69b42224edb1
#5 DONE 0.6s
#3 [internal] load metadata for docker.io/library/alpine:3.13
#3 sha256:5ecd52757bf8256597d3bd40d120a58086e4bc5517721ddf15a4b534c85228a0
#3 DONE 1.4s
#6 [stage-2 1/7] FROM docker.io/library/alpine:3.13@sha256:a75afd8b57e7f34e4dad8d65e2c7ba2e1975c795ce1ee22fa34f8cf46f96a3be
#6 sha256:ac117f80319f3dfc5f3a89bed8fde570a7095ef011b4aa4f1832954fe5fae646
#6 DONE 0.0s
#10 [build 1/14] FROM docker.io/library/golang:1.15-alpine3.13@sha256:a025015951720f3227acd51b0a99a71578b574a63172ea7d2415c60ae5e2bc0a
#10 sha256:945f28b33676aa8e1e61782ed1c14580cb6577c9dae69198435ffaa8642d6acc
#10 DONE 0.0s
#12 [watchdog 1/1] FROM ghcr.io/openfaas/of-watchdog:0.8.4@sha256:9a0b595f77724005ef7336d68a5a6347426d1931c82f238916496f27d7ea2e74
#12 sha256:0b83d079b8d19e155cadf89ef4c3782ffd5a248d98dbd78ee94c18245505455c
#12 DONE 0.0s
#17 [internal] load build context
#17 sha256:22e4a759ec2fb76c5ae310d48e674f49bb5419d3eeca06c2b27c8987047be299
#17 transferring context: 496.45kB done
#17 DONE 0.0s
#20 [build 9/14] RUN if [ -d ./function/vendor ]; then echo "moving handler vendor" && mv -f ./function/vendor .; else echo "using modules or vendor not found "; fi
#20 sha256:59dd69b01bd2f54de6b035b838eaf41765d90591e42e1add8f2597a30122102d
#20 CACHED
#22 [build 11/14] WORKDIR /go/src/handler/function
#22 sha256:0f0ac30cb75d30ea26d73d4c9c2f1fd1e5b0deec9cdb447c7136b29d4beba8f2
#22 CACHED
#18 [build 7/14] COPY . .
#18 sha256:5f280c8982f74eb09a72e9966ab26deb656de679d2275697d355679904bac9c7
#18 CACHED
#16 [build 6/14] WORKDIR /go/src/handler
#16 sha256:2d57cb8be74105b3bf574dde9b3acfb2afb9ef7c3db0babaf2976e90bd189f0a
#16 CACHED
#23 [build 12/14] RUN GOOS=linux GOARCH=amd64 go test ./... -cover
#23 sha256:db1d87d696d6f1822011c62aa89d515dadbf2eb0b3eb304223308a73644ce7ef
#23 CACHED
#8 [stage-2 3/7] RUN mkdir -p /home/app && chown app /home/app
#8 sha256:392db714e3258ded26fd35e24dbf9da4b34d5f1b72bafa6651edb2c67c6e337a
#8 CACHED
#26 [stage-2 5/7] COPY --from=build --chown=app /go/src/handler/handler .
#26 sha256:3d4993c2f2577fd2fb316fc0705cdf656ddb2108d79f88520308dcc9bc09d238
#26 CACHED
#14 [build 4/14] RUN chmod +x /usr/bin/fwatchdog
#14 sha256:0ce54538eaf598232623e0d0131601ae53ed928d280203d11398984ad325296d
#14 CACHED
#21 [build 10/14] RUN test -z "$(gofmt -l $(find . -type f -name '*.go' -not -path "./vendor/*" -not -path "./function/vendor/*"))" || { echo "Run "gofmt -s -w" on your Golang code"; exit 1; }
#21 sha256:700ed50eb6b399e541605fae19f2ba57943b7925d06b01eb79436008cc735ea6
#21 CACHED
#27 [stage-2 6/7] COPY --from=build --chown=app /usr/bin/fwatchdog .
#27 sha256:361a3b5061edb01c935a8e442bfb9bf1cbc4f162ce840bba5b7f2071c3f50c1c
#27 CACHED
#13 [build 3/14] COPY --from=watchdog /fwatchdog /usr/bin/fwatchdog
#13 sha256:3d4b56eeab2f493ee25bcffa0a77d52bf329e8d5bcf1a7f4f9e60d76207b024c
#13 CACHED
#15 [build 5/14] RUN mkdir -p /go/src/handler
#15 sha256:49827b6b29acd15ceb89996f6858464d41ef41d45bf4439caa09ad778a541c9b
#15 CACHED
#11 [build 2/14] RUN apk --no-cache add git
#11 sha256:e895da5279e30155cbfeb55e93d54399f92e5b71ef96937c7955731c811f752e
#11 CACHED
#19 [build 8/14] RUN cat function/GO_REPLACE.txt >> ./go.mod || exit 0
#19 sha256:e8470771778a6d5753f72db5f421edc35846630310fc09596f8dae241fda872d
#19 CACHED
#7 [stage-2 2/7] RUN apk --no-cache add ca-certificates && addgroup -S app && adduser -S -g app app
#7 sha256:f3920e39e6cb6756252ef8376a8a6a562e10f4156f6387b0e5aa5f3204bbbeb5
#7 CACHED
#9 [stage-2 4/7] WORKDIR /home/app
#9 sha256:8d07b104cc02b839e2f8b214eb64d1906c976d87c5178fc380b99ff4ee79f52d
#9 CACHED
#24 [build 13/14] WORKDIR /go/src/handler
#24 sha256:ff1630159ddb306a357f8bcae32cdabc0d1b3852179bfba444e47dba0b644aee
#24 CACHED
#25 [build 14/14] RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build --ldflags "-s -w" -a -installsuffix cgo -o handler .
#25 sha256:ef4ce81a267fab2bf49828dc2184db924b15090ed4b8241a8172996473624d2e
#25 CACHED
#28 [stage-2 7/7] COPY --from=build --chown=app /go/src/handler/function/ .
#28 sha256:18d8c2e6684e3b4a3ead499390a5c901e4dd1438d59332d61b0889d5a412add7
#28 CACHED
#29 exporting to image
#29 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#29 exporting layers done
#29 writing image sha256:7374b12db800120b5c6de39d05bb1abfc3d72edb7b96b09cc22b0b25f72a606f done
#29 naming to docker.io/library/publisher:latest done
#29 DONE 0.0s
Image: publisher:latest built.
[0] < Building publisher done in 2.38s.
[0] Worker done.
Total build time: 2.38s
What about with buildkit disabled?
@alexellis i will post the logs in a moment, but the docker docs are prtty clear about cache invalidation for RUN
statements
The cache for RUN instructions isn’t invalidated automatically during the next build. The cache for an instruction like RUN apt-get dist-upgrade -y will be reused during the next build. The cache for RUN instructions can be invalidated by using the --no-cache flag, for example docker build --no-cache.
The cache for RUN instructions can be invalidated by ADD and COPY instructions.
There is no reason to expect the if/else
to cause a cache invalidation any more than the line immediately before it to copy the go mod replace statements.
Here are the logs when buildkit is disabled
DOCKER_BUILDKIT=0 faas-cli build -f publisher.yml --build-arg GO111MODULE=off
[0] > Building publisher.
Clearing temporary build folder: ./build/publisher/
Preparing: ./publisher/ build/publisher/function
Building: publisher:latest with golang-http template. Please wait..
Sending build context to Docker daemon 569.9kB
Step 1/36 : FROM --platform=${TARGETPLATFORM:-linux/amd64} ghcr.io/openfaas/of-watchdog:0.8.4 as watchdog
---> 4715559e6ad8
Step 2/36 : FROM --platform=${BUILDPLATFORM:-linux/amd64} golang:1.15-alpine3.13 as build
---> b8d8ad7b4ab7
Step 3/36 : ARG TARGETPLATFORM
---> Using cache
---> e54de4e3e329
Step 4/36 : ARG BUILDPLATFORM
---> Using cache
---> 002f6d91ad3b
Step 5/36 : ARG TARGETOS
---> Using cache
---> 003ab4a52c89
Step 6/36 : ARG TARGETARCH
---> Using cache
---> 0978e1807ae2
Step 7/36 : RUN apk --no-cache add git
---> Using cache
---> fd831584b24f
Step 8/36 : COPY --from=watchdog /fwatchdog /usr/bin/fwatchdog
---> Using cache
---> ce43ffb34a27
Step 9/36 : RUN chmod +x /usr/bin/fwatchdog
---> Using cache
---> 26f87ae9e6d1
Step 10/36 : ENV CGO_ENABLED=0
---> Using cache
---> 03e5437e99fc
Step 11/36 : RUN mkdir -p /go/src/handler
---> Using cache
---> 89964a100c26
Step 12/36 : WORKDIR /go/src/handler
---> Using cache
---> 633c9382007b
Step 13/36 : COPY . .
---> Using cache
---> 8e51bfc34593
Step 14/36 : ARG GO111MODULE="off"
---> Using cache
---> 02efddd77e3d
Step 15/36 : ARG GOPROXY=""
---> Using cache
---> 638e050b3780
Step 16/36 : ARG GOFLAGS=""
---> Using cache
---> 2844cdb0943e
Step 17/36 : RUN cat function/GO_REPLACE.txt >> ./go.mod || exit 0
---> Using cache
---> bdce1d761aa5
Step 18/36 : RUN if [ -d ./function/vendor ]; then echo "moving handler vendor" && mv -f ./function/vendor .; else echo "using modules or vendor not found "; fi
---> Using cache
---> ba70a5a38292
Step 19/36 : RUN test -z "$(gofmt -l $(find . -type f -name '*.go' -not -path "./vendor/*" -not -path "./function/vendor/*"))" || { echo "Run \"gofmt -s -w\" on your Golang code"; exit 1; }
---> Using cache
---> ccf7a5efd4ab
Step 20/36 : WORKDIR /go/src/handler/function
---> Using cache
---> 89d0e8bc709e
Step 21/36 : RUN GOOS=${TARGETOS} GOARCH=${TARGETARCH} go test ./... -cover
---> Using cache
---> d94f2ed7084d
Step 22/36 : WORKDIR /go/src/handler
---> Using cache
---> 59aac535b520
Step 23/36 : RUN CGO_ENABLED=${CGO_ENABLED} GOOS=${TARGETOS} GOARCH=${TARGETARCH} go build --ldflags "-s -w" -a -installsuffix cgo -o handler .
---> Using cache
---> 33eff2c16a04
Step 24/36 : FROM --platform=${TARGETPLATFORM:-linux/amd64} alpine:3.13
---> 28f6e2705743
Step 25/36 : RUN apk --no-cache add ca-certificates && addgroup -S app && adduser -S -g app app
---> Using cache
---> 16d4eccf075b
Step 26/36 : RUN mkdir -p /home/app && chown app /home/app
---> Using cache
---> 7d98c7f0dc19
Step 27/36 : WORKDIR /home/app
---> Using cache
---> c9a5f5a1b0c0
Step 28/36 : COPY --from=build --chown=app /go/src/handler/handler .
---> Using cache
---> 9146e303eade
Step 29/36 : COPY --from=build --chown=app /usr/bin/fwatchdog .
---> Using cache
---> 3f68b293e5be
Step 30/36 : COPY --from=build --chown=app /go/src/handler/function/ .
---> Using cache
---> 5f63bdb1bbc1
Step 31/36 : USER app
---> Using cache
---> 75ae534882fd
Step 32/36 : ENV fprocess="./handler"
---> Using cache
---> bb0f94d490d7
Step 33/36 : ENV mode="http"
---> Using cache
---> c51108fc209d
Step 34/36 : ENV upstream_url="http://127.0.0.1:8082"
---> Using cache
---> 114a47a9d7e2
Step 35/36 : ENV prefix_logs="false"
---> Using cache
---> 40b392698500
Step 36/36 : CMD ["./fwatchdog"]
---> Using cache
---> 2fa0b5044715
Successfully built 2fa0b5044715
Successfully tagged publisher:latest
Image: publisher:latest built.
[0] < Building publisher done in 2.99s.
[0] Worker done.
Total build time: 2.99s
After #56 - what's the recommended practice? I'd love to see an example repo that I can go by.
@tatemz are you looking to use Go modules with vendoring or non-modules with vendoring? or no vendoring and just Go modules?
I use faas-cli in a mono repo atm where my functions live in folder with subfolders organized by type. Ideally, I feel that putting a go.mod
in each function handler is an idea situation in order that each function could manage it's own third party deps. That said, I don't like the idea of vendor
directories littered throughout my project, but maybe that's just the way it's gotta be.
In short, I think go modules with no vendoring is what my gut is telling me to be the ideal. 🤷
EDIT: I suppose this would assume GO111MODULE="auto"
or GO111MODULE="on"
, which isn't really addressed by #56 and may not be pertinent to this issue, but maybe something to keep exploring
EDIT 2: I read through #56 more regarding the documentation around sub-packages. I suppose the biggest question I feel is unanswered is how all functions could share a package. Right now the updated documentation gives an example of a sub-package within a function handler, but how could we use a foo-pkg
module in n+
go handlers?
EDIT 3: Answering my own predicament, the solution is to use go modules replace
in conjunction with the faas-cli
YAML configuration for copy
in order to copy common packages to the build directories.
configuration:
copy:
- ./common
it should "just work" if you add the go.mod and go.sum to each function, no vendor folder required.
I think you are right about the mix of "copy" + replace to copy a shared folder from the same monorepo, but i would need to try it to say for sure
Another note. I think this can be more idempotent:
RUN cat function/GO_REPLACE.txt >> ./go.mod || exit 0
With a bit of refactoring:
RUN for replacement in $(cat function/GO_REPLACE.txt); do go mod edit -replace $replacement; done
GO_REPLACE.txt
would have to contain the go mod edit
command with something like:
foobar.com/my/pkg@v0.0.0=./common
This way any replacements that may already exist with that package name would be updated rather than duplicated (which would cause an error). See go mod edit
docs
Thoughts?
@tatemz that is a good idea, with go mod edit
, but we did remove the GO_REPLACE.txt
requirement and the script now has this section that is supposed to rewrite the existing replace statements https://github.com/openfaas/golang-http-template/blob/5d6376e8aa1af5fa347c3205daa90d2050b64091/template/golang-http/modules-cleanup.sh#L44-L50 so that the replace statements in your go.mod
should just work, at least that is the theory.
Looks like we have resolved this issue, so I'm going to close it.
If anyone lands here, needing help. Raise your own issue please.
/lock: resolved
Expected Behaviour
It should work? 🤷♂️
Current Behaviour
Steps to Reproduce (for bugs)
We were following the template documentation guide as described here.
$ faas-cli template store pull golang-http
$ faas-cli new notifier-fn --lang golang-http --prefix Dentrax
$ cd notifier-fn
$ go mod init github.com/Dentrax/foo
handler.go
Context
go.mod
notifier-fn.yml:
What we have tried so far that did not work
Each step we applied these in sequential:
faas-cli up -f notifier-fn.yml
faas-cli up -f notifier-fn.yml --build-arg GO111MODULE=on
Single
handler.go
file:$ go mod init ...
$ go mod vendor
+$ go mod tidy
? (w/vendor
dir)Using
GO_REPLACE.txt
in parent dirYour Environment
FaaS-CLI version
CLI: commit: 6b5e7a14a598527063c7c05fb4187739b6eb6d38 version: 0.13.6Docker version
Client: Docker Engine - Community Cloud integration: 1.0.7 Version: 20.10.2 API version: 1.41 Go version: go1.13.15 Git commit: 2291f61 Built: Mon Dec 28 16:12:42 2020 OS/Arch: darwin/amd64 Context: default Experimental: true Server: Docker Engine - Community Engine: Version: 20.10.2 API version: 1.41 (minimum version 1.12) Go version: go1.13.15 Git commit: 8891c58 Built: Mon Dec 28 16:15:28 2020 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.4.3 GitCommit: 269548fa27e0089a8b8278fc4fc781d7f65a939b runc: Version: 1.0.0-rc92 GitCommit: ff819c7e9184c13b7c2607fe6c30ae19403a7aff docker-init: Version: 0.19.0 GitCommit: de40ad0Are you using Docker Swarm (FaaS-swarm ) or Kubernetes (FaaS-netes)?
NO
Operating System and version (e.g. Linux, Windows, MacOS):
macOS 11.0.1
We are not sure why the given template documentation does not work as we expected. I think we are missing some important detail on the documentation. Actually, the command that failing above is passing on our local workspace:
$ CGO_ENABLED=${CGO_ENABLED} GOOS=${TARGETOS} GOARCH=${TARGETARCH} go build --ldflags "-s -w" -a -installsuffix cgo -o handler .
Any ideas?cc: @developer-guy