search

apache / incubator-streampark

Make stream processing easier! Easy-to-use streaming application development framework and operation platform.
https://streampark.apache.org/
Apache License 2.0
3.86k stars 996 forks source link

[Bug] Kubernetes application release docker build error: layer does not exist #3308

Open tcmtang opened 11 months ago

tcmtang commented 11 months ago

Search before asking

Java Version

java version "1.8.0_211"

Scala Version

2.11.x

StreamPark Version

2.1.1

Flink Version

1.16.2

deploy mode

kubernetes-application

What happened

Release kubernetes application task docker build error: layer does not exist.

Error Stack:

[streampark-build-pipeline-executor-0] ERROR org.apache.streampark.flink.packer.pipeline.impl.FlinkK8sApplicationBuildPipeline - [StreamPark] [streampark-packer] building pipeline has failed. | appName=flink-test
java.util.concurrent.ExecutionException: com.github.dockerjava.api.exception.DockerClientException: Could not build image: failed to export image: failed to create image: failed to get layer sha256:a1462763f944a429da7885970b752a1ca04b6641377275bc3dfa62d61ebb7608: layer does not exist
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:206)
        at org.apache.streampark.flink.packer.pipeline.BuildPipeline.$anonfun$launch$1(BuildPipeline.scala:136)
        at scala.util.Try$.apply(Try.scala:213)
        at org.apache.streampark.flink.packer.pipeline.BuildPipeline.launch(BuildPipeline.scala:129)
        at org.apache.streampark.flink.packer.pipeline.BuildPipeline.launch$(BuildPipeline.scala:127)
        at org.apache.streampark.flink.packer.pipeline.impl.FlinkK8sApplicationBuildPipeline.launch(FlinkK8sApplicationBuildPipeline.scala:42)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: com.github.dockerjava.api.exception.DockerClientException: Could not build image: failed to export image: failed to create image: failed to get layer sha256:a1462763f944a429da7885970b752a1ca04b6641377275bc3dfa62d61ebb7608: layer does not exist
        at com.github.dockerjava.api.command.BuildImageResultCallback.getImageId(BuildImageResultCallback.java:78)
        at com.github.dockerjava.api.command.BuildImageResultCallback.awaitImageId(BuildImageResultCallback.java:50)
        at org.apache.streampark.flink.packer.pipeline.impl.FlinkK8sApplicationBuildPipeline.$anonfun$buildProcess$21(FlinkK8sApplicationBuildPipeline.scala:180)
        at org.apache.streampark.flink.packer.pipeline.impl.FlinkK8sApplicationBuildPipeline.$anonfun$buildProcess$21$adapted(FlinkK8sApplicationBuildPipeline.scala:165)
        at org.apache.streampark.flink.packer.docker.package$.usingDockerClient(package.scala:46)
        at org.apache.streampark.flink.packer.pipeline.impl.FlinkK8sApplicationBuildPipeline.$anonfun$buildProcess$20(FlinkK8sApplicationBuildPipeline.scala:182)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
        at org.apache.streampark.flink.packer.pipeline.BuildPipeline.$anonfun$execStep$1(BuildPipeline.scala:102)
        at scala.util.Try$.apply(Try.scala:213)
        at org.apache.streampark.flink.packer.pipeline.BuildPipeline.execStep(BuildPipeline.scala:97)
        at org.apache.streampark.flink.packer.pipeline.BuildPipeline.execStep$(BuildPipeline.scala:96)
        at org.apache.streampark.flink.packer.pipeline.impl.FlinkK8sApplicationBuildPipeline.execStep(FlinkK8sApplicationBuildPipeline.scala:42)
        at org.apache.streampark.flink.packer.pipeline.impl.FlinkK8sApplicationBuildPipeline.buildProcess(FlinkK8sApplicationBuildPipeline.scala:182)
        at org.apache.streampark.flink.packer.pipeline.impl.FlinkK8sApplicationBuildPipeline.buildProcess(FlinkK8sApplicationBuildPipeline.scala:42)
        at org.apache.streampark.flink.packer.pipeline.BuildPipeline$$anon$1.call(BuildPipeline.scala:134)
        at org.apache.streampark.flink.packer.pipeline.BuildPipeline$$anon$1.call(BuildPipeline.scala:133)
        ... 4 common frames omitted

Dockerfile step:

Step 1/4 : FROM registry-vpc.cn-shenzhen.aliyuncs.com/mmg-sys/flink:1.16.2-java11
Step 2/4 : RUN mkdir -p $FLINK_HOME/usrlib
Step 3/4 : COPY lib $FLINK_HOME/lib/
Step 4/4 : COPY streampark-flinkjob_flink-test.jar $FLINK_HOME/usrlib/streampark-flinkjob_flink-test.jar

Docker version:

Client: Docker Engine - Community
 Version:           23.0.6
 API version:       1.42
 Go version:        go1.19.9
 Git commit:        ef23cbc
 Built:             Fri May  5 21:19:08 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          23.0.6
  API version:      1.42 (minimum version 1.12)
  Go version:       go1.19.9
  Git commit:       9dbdbd4
  Built:            Fri May  5 21:18:15 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Error Exception

No response

Screenshots

No response

Are you willing to submit PR?

Code of Conduct

caicancai commented 9 months ago

I'm very sorry for replying to you so late. Has the bug been resolved?

caicancai commented 9 months ago

Have you filled in the docker information correctly? Did the build docker fail?