sourcenetwork / defradb

DefraDB is a Peer-to-Peer Edge Database. It's the core data storage system for the Source Network Ecosystem, built with IPLD, LibP2P, CRDTs, and Semantic open web properties.
446 stars 44 forks source link

CircleCI permission errors when restoring cache slowing the build to 45minutes. #370

Closed shahzadlone closed 2 years ago

shahzadlone commented 2 years ago

My hunch is that this is happening because when we updated from the legacy docker image (circleci/golang:1.17.5) to the one that supercedes it (cimg/go:1.17) we neede to change the cache key.

I think even just incrementing go-mod-v4-{{ checksum "go.sum" }} to go-mod-v5-{{ checksum "go.sum" }} shoud do the trick so that a new cache is generated (for the new image, as we switched images)

Click to expand and see CircleCi log! [Download the full output as a file](https://circleci.com/api/v1.1/project/github/sourcenetwork/defradb/1884/output/102/0?file=true&allocation-id=62605f3669446077811ae23f-0-build%2F6EF8EEC) 4/web/server/messaging: no such file or directory Unable to set permissions on "/go/pkg/mod/go.uber.org/zap@v1.19.0/zapio" - chmod /go/pkg/mod/go.uber.org/zap@v1.19.0/zapio: no such file or directory Unable to set permissions on "/go/pkg/mod/google.golang.org/api@v0.30.0/run/v1alpha1" - chmod /go/pkg/mod/google.golang.org/api@v0.30.0/run/v1alpha1: no such file or directory Unable to set permissions on "/go/pkg/mod/github.com/apache/thrift@v0.13.0/lib/java/src/org/apache/thrift/async" - chmod /go/pkg/mod/github.com/apache/thrift@v0.13.0/lib/java/src/org/apache/thrift/async: no such file or directory Unable to set permissions on "/go/pkg/mod/github.com/go-playground/locales@v0.13.0/br_FR" - chmod /go/pkg/mod/github.com/go-playground/locales@v0.13.0/br_FR: no such file or directory Unable to set permissions on "/go/pkg/mod/github.com/ipld/go-ipld-prime@v0.11.0/must" - chmod /go/pkg/mod/github.com/ipld/go-ipld-prime@v0.11.0/must: no such file or directory Unable to set permissions on "/go/pkg/mod/github.com/libp2p/go-libp2p-routing@v0.0.1" - chmod /go/pkg/mod/github.com/libp2p/go-libp2p-routing@v0.0.1: no such file or directory Unable to set permissions on "/go/pkg/mod/cache/download/github.com/libp2p/go-libp2p-testing/@v" - chmod /go/pkg/mod/cache/download/github.com/libp2p/go-libp2p-testing/@v: no such file or directory Unable to set permissions on "/go/pkg/mod/google.golang.org/genproto@v0.0.0-20220204002441-d6cc3cc0770e/googleapis/cloud/gkehub/metering/v1alpha" - chmod /go/pkg/mod/google.golang.org/genproto@v0.0.0-20220204002441-d6cc3cc0770e/googleapis/cloud/gkehub/metering/v1alpha: no such file or directory Unable to set permissions on "/go/pkg/mod/github.com/textileio/go-threads@v1.1.2/api" - chmod /go/pkg/mod/github.com/textileio/go-threads@v1.1.2/api: no such file or directory Unable to set permissions on "/go/pkg/mod/cache/download/github.com/improbable-eng" - chmod /go/pkg/mod/cache/download/github.com/improbable-eng: no such file or directory Unable to set permissions on "/go/pkg/mod/cache/download/sumdb/sum.golang.org/lookup/github.com" - chmod /go/pkg/mod/cache/download/sumdb/sum.golang.org/lookup/github.com: no such file or directory Unable to set permissions on "/go/pkg/mod/github.com/btcsuite/btcd@v0.22.0-beta/blockchain/fullblocktests" - chmod /go/pkg/mod/github.com/btcsuite/btcd@v0.22.0-beta/blockchain/fullblocktests: no such file or directory Unable to set permissions on "/go/pkg/mod/github.com/ipld/go-ipld-prime@v0.11.0/traversal/selector" - chmod /go/pkg/mod/github.com/ipld/go-ipld-prime@v0.11.0/traversal/selector: no such file or directory Unable to set permissions on "/go/pkg/mod/github.com/envoyproxy/go-control-plane@v0.9.10-0.20210907150352-cf90f659a021/envoy/config/metrics" - chmod /go/pkg/mod/github.com/envoyproxy/go-control-plane@v0.9.10-0.20210907150352-cf90f659a021/envoy/config/metrics: no such file or directory Unable to set permissions on "/go/pkg/mod/github.com/google/martian@v2.1.0+incompatible/martianurl" - chmod /go/pkg/mod/github.com/google/martian@v2.1.0+incompatible/martianurl: no such file or directory Unable to set permissions on "/go/pkg/mod/github.com/ipfs/go-ds-leveldb@v0.5.0" - chmod /go/pkg/mod/github.com/ipfs/go-ds-leveldb@v0.5.0: no such file or directory /go/pkg/mod/google.golang.org/api@v0.30.0/accesscontextmanager/v1: no such file or directory Unable to set permissions on "/go/pkg/mod/cache/download/sumdb/sum.golang.org/lookup/github.com/dop251" - chmod /go/pkg/mod/cache/download/sumdb/sum.golang.org/lookup/github.com/dop251: no such file or directory Unable to set permissions on "/go/pkg/mod/cloud.google.com/go@v0.65.0/pubsublite/apiv1" - chmod /go/pkg/mod/cloud.google.com/go@v0.65.0/pubsublite/apiv1: no such file or directory Unable to set permissions on "/go/pkg/mod/git.apache.org/thrift.git@v0.0.0-20180902110319-2566ecd5d999/lib/php/lib" - chmod /go/pkg/mod/git.apache.org/thrift.git@v0.0.0-20180902110319-2566ecd5d999/lib/php/lib: no such file or directory Unable to set permissions on "/go/pkg/mod/github.com/cheekybits/genny@v1.0.0/parse/test/queue" - chmod /go/pkg/mod/github.com/cheekybits/genny@v1.0.0/parse/test/queue: no such file or directory Unable to set permissions on "/go/pkg/mod/github.com/aws/aws-sdk-go@v1.27.0/service/accessanalyzer" - chmod /go/pkg/mod/github.com/aws/aws-sdk-go@v1.27.0/service/accessanalyzer: no such file or directory Unable to set permissions on "/go/pkg/mod/honnef.co/go/tools@v0.0.1-2020.1.4/structlayout" - chmod /go/pkg/mod/honnef.co/go/tools@v0.0.1-2020.1.4/structlayout: no such file or directory Unable to set permissions on "/go/pkg/mod/golang.org/x/oauth2@v0.0.0-20210514164344-f6687ab2804c/yandex" - chmod /go/pkg/mod/golang.org/x/oauth2@v0.0.0-20210514164344-f6687ab2804c/yandex: no such file or directory Unable to set permissions on "/go/pkg/mod/cache/download/github.com/armon" - chmod /go/pkg/mod/cache/download/github.com/armon: no such file or directory Unable to set permissions on "/go/pkg/mod/cache/download/github.com/libp2p/go-libp2p-interface-pnet" - chmod /go/pkg/mod/cache/download/github.com/libp2p/go-libp2p-interface-pnet: no such file or directory Unable to set permissions on "/go/pkg/mod/git.apache.org/thrift.git@v0.0.0-20180902110319-2566ecd5d999/lib/haxe/src/org/apache/thrift/protocol" - chmod /go/pkg/mod/git.apache.org/thrift.git@v0.0.0-20180902110319-2566ecd5d999/lib/haxe/src/org/apache/thrift/protocol: no such file or directory Unable to set permissions on "/go/pkg/mod/github.com/hashicorp/go.net@v0.0.1/internal" - chmod /go/pkg/mod/github.com/hashicorp/go.net@v0.0.1/internal: no such file or directory Unable to set permissions on "/go/pkg/mod/github.com/aws/aws-sdk-go-v2@v0.18.0/models/apis/athena" - chmod /go/pkg/mod/github.com/aws/aws-sdk-go-v2@v0.18.0/models/apis/athena: no such file or directory Unable to set permissions on "/go/pkg/mod/golang.org/x/crypto@v0.0.0-20210921155107-089bfa567519/openpgp/s2k" - chmod /go/pkg/mod/golang.org/x/crypto@v0.0.0-20210921155107-089bfa567519/openpgp/s2k: no such file or directory Unable to set permissions on "/go/pkg/mod/cache/download/github.com/aws/aws-sdk-go/@v" - chmod /go/pkg/mod/cache/download/github.com/aws/aws-sdk-go/@v: no such file or directory Unable to set permissions on "/go/pkg/mod/cache/download/github.com/golang/glog/@v" - chmod /go/pkg/mod/cache/download/github.com/golang/glog/@v: no such file or directory Unable to set permissions on "/go/pkg/mod/go.etcd.io/etcd@v0.0.0-20191023171146-3cf2f69b5738/client/integration" - chmod /go/pkg/mod/go.etcd.io/etcd@v0.0.0-20191023171146-3cf2f69b5738/client/integration: no such file or directory Unable to set permissions on "/go/pkg/mod/git.apache.org/thrift.git@v0.0.0-20180902110319-2566ecd5d999/lib/ocaml" - chmod /go/pkg/mod/git.apache.org/thrift.git@v0.0.0-20180902110319-2566ecd5d999/lib/ocaml: no such file or directory Unable to set permissions on "/go/pkg/mod/github.com/google/flatbuffers@v1.12.1/tests/rust_usage_test" - chmod /go/pkg/mod/github.com/google/flatbuffers@v1.12.1/tests/rust_usage_test: no such file or directory Unable to set permissions on "/go/pkg/mod/google.golang.org/api@v0.30.0/doubleclickbidmanager/v1" - chmod /go/pkg/mod/google.golang.org/api@v0.30.0/doubleclickbidmanager/v1: no such file or directory Unable to set permissions on "/go/pkg/mod/cache/download/github.com/cncf/udpa/go/@v" - chmod /go/pkg/mod/cache/download/github.com/cncf/udpa/go/@v: no such file or directory Unable to set permissions on "/go/pkg/mod/github.com/envoyproxy/go-control-plane@v0.9.10-0.20210907150352-cf90f659a021/envoy/service/cluster" - chmod /go/pkg/mod/github.com/envoyproxy/go-control-plane@v0.9.10-0.20210907150352-cf90f659a021/envoy/service/cluster: no such file or directory Unable to set permissions on "/go/pkg/mod/github.com/libp2p/go-libp2p-kbucket@v0.4.7/peerdiversity" - chmod /go/pkg/mod/github.com/libp2p/go-libp2p-kbucket@v0.4.7/peerdiversity: no such file or directory Unable to set permissions on "/go/pkg/mod/google.golang.org/genproto@v0.0.0-20220204002441-d6cc3cc0770e/googleapis/devtools/clouderrorreporting" - chmod /go/pkg/mod/google.golang.org/genproto@v0.0.0-20220204002441-d6cc3cc0770e/googleapis/devtools/clouderrorreporting: no such file or directory Unable to set permissions on "/go/pkg/mod/cache/download/github.com/spf13/jwalterweatherman/@v" - chmod /go/pkg/mod/cache/download/github.com/spf13/jwalterweatherman/@v: no such file or directory
shahzadlone commented 2 years ago

https://github.com/sourcenetwork/defradb/pull/371 Fixes this.

However for future be mindful that if this happens again it might be due to another 1.17.x image being pulled down automatically however I think circleci devs have had done some magic to ensure that doesn't happen because they don't recommend pinning a specific variant. Just a heads up in future if it does happen again (where we didn't change the images), then we can either: 1) fix the full image, i.e. : cimg/go:1.17.19 OR, 2) pass a environment variable that would set the go image, and use the same environment variable to be part of the cache restore/save key.