search

moby / buildkit

concurrent, cache-efficient, and Dockerfile-agnostic builder toolkit
https://github.com/moby/moby/issues/34227
Apache License 2.0
8.21k stars 1.16k forks source link

ERROR: error writing manifest blob: failed commit on ref unexpected status: 404 Not Found #3781

Open renhao-0518 opened 1 year ago

renhao-0518 commented 1 year ago

Dockerfile

FROM harbor-buildkit.test.net/common/busybox:1.34.1
ADD search-entry-service.jar /opt/apps/
RUN mkdir aa

Buildkitd

  containers:
  - args:
    - --addr
    - unix:///run/user/1000/buildkit/buildkitd.sock
    - --addr
    - tcp://0.0.0.0:1234
    - --oci-worker-no-process-sandbox
    image: harbor-buildkit.test.net/common/buildkit:v0.11.5-rootless

Command

 - name: build-and-push
    image: harbor-buildkit.test.net/common/buildkit:v0.11.5
    workingDir: /workspace
    env:
      - name: "DOCKER_CONFIG"
        value: "/tekton/home/.docker/"
      - name: BUILDKIT_COLORS
        value: "run=123,20,245:error=yellow:cancel=blue:warning=white"
    volumeMounts:
      - name: docker-secret
        mountPath: /tekton/home/.docker/
    command:
    - buildctl
    args:
    - --debug
    - --addr=tcp://buildkitd.buildkit:1234
    - build
    - --progress=plain
    - --frontend=dockerfile.v0
    - --local
    - context=.
    - --local
    - dockerfile=.
    - --output
    - type=image,name=harbor-buildkit.test.net/common/buildkit-test:v1,push=true
    - --export-cache
    - type=registry,ref=harbor-buildkit.test.net/cache/buildcache
    - --import-cache
    - type=registry,ref=harbor-buildkit.test.net/cache/buildcache

Log

time="2023-04-07T03:51:53Z" level=debug msg="serving grpc connection" spanID=95c6746122b94277 traceID=f5d45306f087da2a377aa695e9f83e84
#1 [internal] load .dockerignore
#1 transferring context: 2B done
#1 DONE 0.0s

#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 153B done
#2 DONE 0.0s

#3 [auth] common/busybox:pull token for [harbor-buildkit.test.net](http://harbor-buildkit.test.net/)
#3 DONE 0.0s

#4 [internal] load metadata for [harbor-buildkit.test.net/common/busybox:1.34.1](http://harbor-buildkit.test.net/common/busybox:1.34.1)
#4 DONE 0.2s

#5 [internal] load build context
#5 DONE 0.0s

#6 [1/3] FROM [harbor-buildkit.test.net/common/busybox:1.34.1@sha256:50e44504ea4f19f141118a8a8868e6c5bb9856efa33f2183f5ccea7ac62aacc9](http://harbor-buildkit.test.net/common/busybox:1.34.1@sha256:50e44504ea4f19f141118a8a8868e6c5bb9856efa33f2183f5ccea7ac62aacc9)
#6 resolve [harbor-buildkit.test.net/common/busybox:1.34.1@sha256:50e44504ea4f19f141118a8a8868e6c5bb9856efa33f2183f5ccea7ac62aacc9](http://harbor-buildkit.test.net/common/busybox:1.34.1@sha256:50e44504ea4f19f141118a8a8868e6c5bb9856efa33f2183f5ccea7ac62aacc9) done
#6 DONE 0.0s

#7 [auth] cache/buildcache:pull token for [harbor-buildkit.shizhuang-inc.net](http://harbor-buildkit.shizhuang-inc.net/)
#7 DONE 0.0s

#8 importing cache manifest from [harbor-buildkit.shizhuang-inc.net/cache/buildcache](http://harbor-buildkit.shizhuang-inc.net/cache/buildcache)
#8 ERROR: failed to configure registry cache importer: [harbor-buildkit.shizhuang-inc.net/cache/buildcache:latest:](http://harbor-buildkit.shizhuang-inc.net/cache/buildcache:latest:) not found

#5 [internal] load build context
#5 transferring context: 45B done
#5 DONE 0.0s

#9 [2/3] ADD search-entry-service.jar /opt/apps/
#9 CACHED

#10 [3/3] RUN  mkdir aa
#10 DONE 0.1s

#11 exporting to image
#11 exporting layers
#11 exporting layers 0.2s done
#11 exporting manifest sha256:dbf247b50785df03dedc5eb4e2b1d95d4e6eed2300ca3ec148e19bc53d1c155d done
#11 exporting config sha256:1b61cb83ec3318df9c68ee2b345a074a33d2050261d6333130a37b8c64577599 done
#11 pushing layers 0.1s done
#11 pushing manifest for [harbor-buildkit.shizhuang-inc.net/common/buildkit-test:v1@sha256:dbf247b50785df03dedc5eb4e2b1d95d4e6eed2300ca3ec148e19bc53d1c155d](http://harbor-buildkit.shizhuang-inc.net/common/buildkit-test:v1@sha256:dbf247b50785df03dedc5eb4e2b1d95d4e6eed2300ca3ec148e19bc53d1c155d) 0.1s done
#11 DONE 0.3s

#12 [auth] common/buildkit-test:pull,push token for [harbor-buildkit.shizhuang-inc.net](http://harbor-buildkit.shizhuang-inc.net/)
#12 DONE 0.0s

#13 exporting content cache
#13 preparing build cache for export
#13 ...

#14 [auth] cache/buildcache:pull,push token for [harbor-buildkit.shizhuang-inc.net](http://harbor-buildkit.shizhuang-inc.net/)
#14 DONE 0.0s

#13 exporting content cache
#13 writing layer sha256:3cb635b06aa273034d7080e0242e4b6628c59347d6ddefff019bfd82f45aa7d5 0.1s done
#13 writing layer sha256:76b721c0cdd34aaddb4d3f691aff2819ad59ea5ae9f1fd1deeb067c6f3733b13 0.0s done
#13 writing layer sha256:c7c0f0fdae1f5a0c8c0a3656bd3df6cd3a9ab9b70f3d021455744fe2c04ad61b
time="2023-04-07T03:51:54Z" level=debug msg="stopping session" spanID=95c6746122b94277 traceID=f5d45306f087da2a377aa695e9f83e84
#13 preparing build cache for export 0.2s done
#13 writing layer sha256:c7c0f0fdae1f5a0c8c0a3656bd3df6cd3a9ab9b70f3d021455744fe2c04ad61b 0.0s done
#13 writing config sha256:4cd0734f8d06189d167e2691e6df16d7146c4e9d84f81b7fb9c55284c1c598c2 0.0s done
#13 writing manifest sha256:ef7e30c458f21487380dbbd84bfd4e5a0c2b14abd9dd1b53cac60aa333a08741 0.0s done
#13 ERROR: error writing manifest blob: failed commit on ref "sha256:ef7e30c458f21487380dbbd84bfd4e5a0c2b14abd9dd1b53cac60aa333a08741": unexpected status: 404 Not Found
------
 > importing cache manifest from [harbor-buildkit.shizhuang-inc.net/cache/buildcache:](http://harbor-buildkit.shizhuang-inc.net/cache/buildcache:)
------
------
 > exporting content cache:
------
error: failed to solve: error writing manifest blob: failed commit on ref "sha256:ef7e30c458f21487380dbbd84bfd4e5a0c2b14abd9dd1b53cac60aa333a08741": unexpected status: 404 Not Found
25 v0.11.5 buildkitd --addr unix:///run/user/1000/buildkit/buildkitd.sock --addr tcp://0.0.0.0:1234 --oci-worker-no-process-sandbox
[github.com/moby/buildkit/cache/remotecache.(*contentCacheExporter](http://github.com/moby/buildkit/cache/remotecache.(*contentCacheExporter)).Finalize
    /src/cache/remotecache/export.go:140
[github.com/moby/buildkit/solver/llbsolver.runCacheExporters.func1.1.1](http://github.com/moby/buildkit/solver/llbsolver.runCacheExporters.func1.1.1)
    /src/solver/llbsolver/solver.go:614
[github.com/moby/buildkit/solver/llbsolver.inBuilderContext.func1](http://github.com/moby/buildkit/solver/llbsolver.inBuilderContext.func1)
    /src/solver/llbsolver/solver.go:922
[github.com/moby/buildkit/solver.(*Job](http://github.com/moby/buildkit/solver.(*Job)).InContext
    /src/solver/jobs.go:610
[github.com/moby/buildkit/solver/llbsolver.inBuilderContext](http://github.com/moby/buildkit/solver/llbsolver.inBuilderContext)
    /src/solver/llbsolver/solver.go:918
[github.com/moby/buildkit/solver/llbsolver.runCacheExporters.func1.1](http://github.com/moby/buildkit/solver/llbsolver.runCacheExporters.func1.1)
    /src/solver/llbsolver/solver.go:595
[golang.org/x/sync/errgroup.(*Group](http://golang.org/x/sync/errgroup.(*Group)).Go.func1
    /src/vendor/golang.org/x/sync/errgroup/errgroup.go:75
runtime.goexit
    /usr/local/go/src/runtime/asm_amd64.s:1594

19 v0.11.5 buildctl --debug --addr=tcp://buildkitd.buildkit:1234 build --progress=plain --frontend=dockerfile.v0 --local context=. --local dockerfile=. --output type=image,[name=harbor-buildkit.shizhuang-inc.net/common/buildkit-test:v1,push=true](http://name%3Dharbor-buildkit.shizhuang-inc.net/common/buildkit-test:v1,push=true) --export-cache type=registry,[ref=harbor-buildkit.shizhuang-inc.net/cache/buildcache](http://ref%3Dharbor-buildkit.shizhuang-inc.net/cache/buildcache) --import-cache type=registry,[ref=harbor-buildkit.shizhuang-inc.net/cache/buildcache](http://ref%3Dharbor-buildkit.shizhuang-inc.net/cache/buildcache)
[github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryClient.func1.1.1](http://github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryClient.func1.1.1)
    /src/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:72
[go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc.UnaryClientInterceptor.func1](http://go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc.UnaryClientInterceptor.func1)
    /src/vendor/go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc/interceptor.go:99
[github.com/moby/buildkit/client.filterInterceptor.func1](http://github.com/moby/buildkit/client.filterInterceptor.func1)
    /src/client/client.go:294
[github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryClient.func1.1.1](http://github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryClient.func1.1.1)
    /src/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:72
[github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryClient.func1](http://github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryClient.func1)
    /src/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:81
[google.golang.org/grpc.(*ClientConn](http://google.golang.org/grpc.(*ClientConn)).Invoke
    /src/vendor/google.golang.org/grpc/call.go:35
[github.com/moby/buildkit/api/services/control.(*controlClient](http://github.com/moby/buildkit/api/services/control.(*controlClient)).Solve
    /src/api/services/control/control.pb.go:2208
[github.com/moby/buildkit/client.(*Client](http://github.com/moby/buildkit/client.(*Client)).solve.func2
    /src/client/solve.go:258
[golang.org/x/sync/errgroup.(*Group](http://golang.org/x/sync/errgroup.(*Group)).Go.func1
    /src/vendor/golang.org/x/sync/errgroup/errgroup.go:75
runtime.goexit
    /usr/local/go/src/runtime/asm_amd64.s:1594

19 v0.11.5 buildctl --debug --addr=tcp://buildkitd.buildkit:1234 build --progress=plain --frontend=dockerfile.v0 --local context=. --local dockerfile=. --output type=image,[name=harbor-buildkit.shizhuang-inc.net/common/buildkit-test:v1,push=true](http://name%3Dharbor-buildkit.shizhuang-inc.net/common/buildkit-test:v1,push=true) --export-cache type=registry,[ref=harbor-buildkit.shizhuang-inc.net/cache/buildcache](http://ref%3Dharbor-buildkit.shizhuang-inc.net/cache/buildcache) --import-cache type=registry,[ref=harbor-buildkit.shizhuang-inc.net/cache/buildcache](http://ref%3Dharbor-buildkit.shizhuang-inc.net/cache/buildcache)
[github.com/moby/buildkit/client.(*Client](http://github.com/moby/buildkit/client.(*Client)).solve.func2
    /src/client/solve.go:273
[golang.org/x/sync/errgroup.(*Group](http://golang.org/x/sync/errgroup.(*Group)).Go.func1
    /src/vendor/golang.org/x/sync/errgroup/errgroup.go:75
runtime.goexit
    /usr/local/go/src/runtime/asm_amd64.s:1594

Version

BuildKit v0.11.5 Buildkitd v.0.11.5-rootless Harbor v2.7.1

imeoer commented 1 year ago

Maybe try the https://github.com/moby/buildkit/pull/3724 or use the inline cache.

dongzhiwei-git commented 1 week ago

Has this problem been resolved