search

docker / build-push-action

GitHub Action to build and push Docker images with Buildx
https://github.com/marketplace/actions/build-and-push-docker-images
Apache License 2.0
4.13k stars 532 forks source link

buildx failed with: ERROR: failed to solve: process ... did not complete successfully: exit code: 100 #933

Closed RunningLeon closed 11 months ago

RunningLeon commented 11 months ago

Contributing guidelines

I've found a bug, and:

Description

Failed to build docker image with 

NVIDIA/nvidia-docker#4 4.944 Fetched 27.9 MB in 4s (7434 kB/s)
NVIDIA/nvidia-docker#4 4.944 Reading package lists...
NVIDIA/nvidia-docker#4 6.737 E: Failed to fetch https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/Packages.gz  File has unexpected size (1127072 != 1126689). Mirror sync in progress? [IP: 152.195.19.142 443]
NVIDIA/nvidia-docker#4 6.737    Hashes of expected file:
NVIDIA/nvidia-docker#4 6.737     - Filesize:1126689 [weak]
NVIDIA/nvidia-docker#4 6.737     - SHA256:7cee2584ca6d97b2f07018ba4f9c3c473fb4e299ed170968a1f8c99c090cc59f
NVIDIA/nvidia-docker#4 6.737     - SHA1:4ee24fac5518a3fcc3702590a0dab32c95484c54 [weak]
NVIDIA/nvidia-docker#4 6.737     - MD5Sum:593faff511765d11055c9919bf2e3bf8 [weak]
NVIDIA/nvidia-docker#4 6.737    Release file created at: Thu, 17 Aug 2023 19:03:05 +0000
NVIDIA/nvidia-docker#4 6.737 E: Some index files failed to download. They have been ignored, or old ones used instead.
NVIDIA/nvidia-docker#4 ERROR: process "/bin/sh -c apt-get update && apt-get install -y --no-install-recommends     rapidjson-dev libgoogle-glog-dev gdb      && rm -rf /var/lib/apt/lists/*" did not complete successfully: exit code: 100
------
 > [2/5] RUN apt-get update && apt-get install -y --no-install-recommends     rapidjson-dev libgoogle-glog-dev gdb      && rm -rf /var/lib/apt/lists/*:
3.275 Get:20 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [28.6 kB]

6.737 E: Failed to fetch https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/Packages.gz  File has unexpected size (1127072 != 1126689). Mirror sync in progress? [IP: 152.195.19.142 443]
6.737    Hashes of expected file:
6.737     - Filesize:1126689 [weak]
6.737     - SHA256:7cee2584ca6d97b2f07018ba4f9c3c473fb4e299ed170968a1f8c99c090cc59f
6.737     - SHA1:4ee24fac5518a3fcc[370](https://github.com/RunningLeon/lmdeploy/actions/runs/5898044107/job/15998515422#step:11:374)2590a0dab32c95484c54 [weak]
6.737     - MD5Sum:593faff511765d11055c9919bf2e3bf8 [weak]
6.737    Release file created at: Thu, 17 Aug 2023 19:03:05 +0000
6.737 E: Some index files failed to download. They have been ignored, or old ones used instead.
------
./docker/Dockerfile:3
--------------------
   2 |     
   3 | >>> RUN apt-get update && apt-get install -y --no-install-recommends \
   4 | >>>     rapidjson-dev libgoogle-glog-dev gdb  \
   5 | >>>     && rm -rf /var/lib/apt/lists/*
   6 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c apt-get update && apt-get install -y --no-install-recommends     rapidjson-dev libgoogle-glog-dev gdb      && rm -rf /var/lib/apt/lists/*" did not complete successfully: exit code: 100
Error: buildx failed with: ERROR: failed to solve: process "/bin/sh -c apt-get update && apt-get install -y --no-install-recommends     rapidjson-dev libgoogle-glog-dev gdb      && rm -rf /var/lib/apt/lists/*" did not complete successfully: exit code: 100

Expected behaviour

Build docker image

Actual behaviour

Failed with building docker image

Repository URL

No response

Workflow run URL

https://github.com/RunningLeon/lmdeploy/actions/runs/5898044107/job/15998515422

YAML workflow

https://github.com/RunningLeon/lmdeploy/actions/runs/5898044107/workflow

Workflow logs

logs_38.zip

BuildKit logs

No response

Additional info

No response

crazy-max commented 11 months ago

Failed to build docker image with

Does not look to be a bug with buildx but apt repo https://developer.download.nvidia.com

RunningLeon commented 11 months ago

hi, thanks for your reply. But we can build successfully build the image on the local Ubuntu machine

crazy-max commented 11 months ago

But we can build successfully build the image on the local Ubuntu machine

Maybe because packages are cached and therefore you can't repro. Try with apt-get clean && apt-get update to make sure.

File has unexpected size (1127072 != 1126689). Mirror sync in progress? [IP: 152.195.19.142 443]

This sounds like a temporary repository server problem that should go away on its own.

RunningLeon commented 11 months ago

hi, changed and still not working

NVIDIA/nvidia-docker#4 5.279 E: Failed to fetch https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/Packages.gz  File has unexpected size (1127072 != 1126689). Mirror sync in progress? [IP: 152.195.19.142 443]
NVIDIA/nvidia-docker#4 5.279    Hashes of expected file:
NVIDIA/nvidia-docker#4 5.279     - Filesize:1126689 [weak]
NVIDIA/nvidia-docker#4 5.279     - SHA256:7cee2584ca6d97b2f07018ba4f9c3c473fb4e299ed170968a1f8c99c090cc59f
NVIDIA/nvidia-docker#4 5.279     - SHA1:4ee24fac5518a3fcc3702590a0dab32c95484c54 [weak]
NVIDIA/nvidia-docker#4 5.279     - MD5Sum:593faff511765d11055c9919bf2e3bf8 [weak]
NVIDIA/nvidia-docker#4 5.279    Release file created at: Thu, 17 Aug 2023 19:03:05 +0000
NVIDIA/nvidia-docker#4 5.279 E: Some index files failed to download. They have been ignored, or old ones used instead.
NVIDIA/nvidia-docker#4 ERROR: process "/bin/sh -c apt-get clean && apt-get update && apt-get install -y --no-install-recommends     rapidjson-dev libgoogle-glog-dev gdb      && rm -rf /var/lib/apt/lists/*" did not complete successfully: exit code: 100
------
 > [2/5] RUN apt-get clean && apt-get update && apt-get install -y --no-install-recommends     rapidjson-dev libgoogle-glog-dev gdb      && rm -rf /var/lib/apt/lists/*:
2.961 Get:20 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1092 kB]

5.279 E: Failed to fetch https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/Packages.gz  File has unexpected size (1127072 != 1126689). Mirror sync in progress? [IP: 152.195.19.142 443]
5.279    Hashes of expected file:
5.279     - Filesize:1126689 [weak]
5.279     - SHA256:7cee2584ca6d97b2f07018ba4f9c3c473fb4e299ed170968a1f8c99c090cc59f
5.279     - SHA1:4ee24fac5518a3fcc[370](https://github.com/RunningLeon/lmdeploy/actions/runs/5901187377/job/16006876421#step:11:374)2590a0dab32c95484c54 [weak]
5.279     - MD5Sum:593faff511765d11055c9919bf2e3bf8 [weak]
5.279    Release file created at: Thu, 17 Aug 2023 19:03:05 +0000
5.279 E: Some index files failed to download. They have been ignored, or old ones used instead.
------
./docker/Dockerfile:3
--------------------
   2 |     
   3 | >>> RUN apt-get clean && apt-get update && apt-get install -y --no-install-recommends \
   4 | >>>     rapidjson-dev libgoogle-glog-dev gdb  \
   5 | >>>     && rm -rf /var/lib/apt/lists/*
   6 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c apt-get clean && apt-get update && apt-get install -y --no-install-recommends     rapidjson-dev libgoogle-glog-dev gdb      && rm -rf /var/lib/apt/lists/*" did not complete successfully: exit code: 100
Error: buildx failed with: ERROR: failed to solve: process "/bin/sh -c apt-get clean && apt-get update && apt-get install -y --no-install-recommends     rapidjson-dev libgoogle-glog-dev gdb      && rm -rf /var/lib/apt/lists/*" did not complete successfully: exit code: 100
guzman109 commented 11 months ago

But we can build successfully build the image on the local Ubuntu machine

Maybe because packages are cached and therefore you can't repro. Try with apt-get clean && apt-get update to make sure.

File has unexpected size (1127072 != 1126689). Mirror sync in progress? [IP: 152.195.19.142 443]

This sounds like a temporary repository server problem that should go away on its own.

They meant to do this on your local computer to reproduce.

crazy-max commented 11 months ago

You probably can't repro locally because this is not using the same CDN as the one in GitHub infrastructure. After investigation, I found this page https://developer.nvidia.com/blog/updating-the-cuda-linux-gpg-repository-key/ and looking at the "File has unexpected size" section, it says to report this issue to NVIDIA:

{{Packages.gz File has unexpected size (631054 != 481481). Mirror sync in progress? [IP: XXX.XXX.XXX.XXX 443]
Hashes of expected file:

* Filesize:481481 [weak]
* SHA256:8556d67c6d380c957f05057f448d994584a135d7ed75e5ae6bb25c3fc1070b0b
* SHA1:c5ea9556407a3b5daec4aac530cd038e9b490441 [weak]
* MD5Sum:a5513131dbd2d4e50f185422ebb43ac9 [weak]
* Release file created at: Mon, 25 Apr 2022 23:27:19 +0000
* E: Some index files failed to download. They have been ignored, or old ones used instead.}}

Solution: Report CDN issue to NVIDIA.

Similar to:

Suggest to open an issue on https://github.com/NVIDIA/nvidia-docker

RunningLeon commented 11 months ago

@crazy-max hi, many thanks for your efforts. Found same issue https://github.com/NVIDIA/cuda-repo-management/issues/10 and solved by adding rm /etc/apt/sources.list.d/cuda*.list in Dockefile