sealer images miss registry docker layers

xiaoleyang2018 commented 1 year ago

What happen?

打集群镜像的时候，结果中随机丢失镜像仓库中的个别镜像的layer层，例如本例中缺少bitnami_kubernetes-event-exporterbitnami_kubernetes-event-exporter 的 _layers层，缺少哪个镜像的layer层是随机发生，不是固定的，个别时候也能打出完整的镜像

├── bitnami_kubernetes-event-exporter │ └── _manifests │ ├── revisions │ │ └── sha256 │ │ └── 5699a070a086d9b61c6c90781dc5e4d11ef042fc171008a725ccd062c74e2d87 │ │ └── link │ └── tags │ └── 1.1.0-debian-11-r25 │ ├── current │ │ └── link │ └── index │ └── sha256 │ └── 5699a070a086d9b61c6c90781dc5e4d11ef042fc171008a725ccd062c74e2d87 │ └── link ├── busybox │ ├── _layers │ │ └── sha256 │ │ ├── 9bf1b624e59da5e71e18c45607c874e315de6155d0094fe4d4daa9361dd7e2c4 │ │ │ └── link │ │ └── ce2f991d7b899f573f7570b836ae8afab15f0b700a5f9a3cd73c9f569d60fc1a │ │ └── link │ ├── _manifests │ │ ├── revisions │ │ │ └── sha256 │ │ │ └── 9421d4cc473b282bf48888476604444fa3c74f6e3f5cddfa210e773c534cd33d │ │ │ └── link │ │ └── tags │ │ └── stable │ │ ├── current │ │ │ └── link │ │ └── index │ │ └── sha256 │ │ └── 9421d4cc473b282bf48888476604444fa3c74f6e3f5cddfa210e773c534cd33d │ │ └── link │ └── _uploads

Relevant log output?

No response

What you expected to happen?

No response

How to reproduce it (as minimally and precisely as possible)?

FROM scratch
COPY arm64/docker cri/docker
COPY arm64/docker-compose cri/docker-compose
COPY arm64/nvidia-container-runtime cri/nvidia-container-runtime
COPY arm64/nvidia-container-toolkit cri/nvidia-container-toolkit
COPY arm64/ascend-docker-runtime cri/ascend-docker-runtime
COPY arm64/k3s/k3s bin/k3s
COPY arm64/sealer/seautil bin/seautil
COPY arm64/helm/helm bin/helm
COPY arm64/helm/helm bin/helmfile
COPY arm64/xfactor/xfactor bin/xfactor
COPY arm64/mig-parted tools/mig-parted
COPY etc etc
COPY arm64/k3s/k3s-airgap-images.tar images/k3s-airgap-images.tar
COPY arm64/registry/registry.tar.gz images/registry.tar.gz
COPY manifests manifests
COPY manifests/imageList.arm64 manifests/imageList
COPY scripts scripts
COPY plugins plugins
COPY charts charts
#CNI calico local://charts/calico
APP local-path-provisioner local://charts/local-path-provisioner
APP gpu-feature-discovery local://charts/gpu-feature-discovery
APP nvidia-device-plugin local://charts/nvidia-device-plugin
APP accelerator-devices-plugin-huawei local://charts/accelerator-devices-plugin-huawei
APP ingress local://charts/ingress
APP cert-manager local://charts/cert-manager
APP kube-prometheus-stack local://charts/kube-prometheus-stack
APP loki-stack local://charts/loki-stack
APP prometheus-adapter local://charts/prometheus-adapter
APP kubernetes-event-exporter local://charts/kubernetes-event-exporter

Anything else we need to know?

之前没加application的时候没发生随机缺少镜像的问题，加了之后需要的镜像个数明显增加。在其他x86的机器上也出现过随机缺少镜像内容的问题

What is the version of Sealer you using?

v0.9.2

What is your OS environment?

NAME="CentOS Linux" VERSION="7 (AltArch)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (AltArch)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/" CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7"

What is the Kernel version?

Linux localhost.domain.com 4.14.0-115.el7a.0.1.aarch64 #1 SMP Sun Nov 25 20:54:21 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux

Other environment you want to tell us?

Cloud provider or hardware configuration:
Install tools:
Others:

starnop commented 1 year ago

Could you please provide the containerImageList from sealer inspect and check to see if it contains the image you mentioned?

xiaoleyang2018 commented 1 year ago

看过临时生成的那个dockerfile，是正常的，sealer build的时候加上-d ,打很多dubeg日志的时候，每次都能打完整镜像。不知道要排查哪里？

xiaoleyang2018 commented 1 year ago

把参数 maxPullGoroutineNum改成 1，问题解决了

kakaZhou719 commented 1 year ago

@xiaoleyang2018 , this pr fixed this bug, FYI: https://github.com/sealerio/sealer/pull/2238/files

sealerio / sealer