pull image error in proxy mode and blockdev mode when convert snapshotter from overlayfs to nydus

[ChengyuZhu6]

After creating a pod using containerd and overlayfs, then subsequently deleting the pod, I switch the snapshotter to Nydus. Upon pulling up nydus-snapshotter and attempting to access the same entry, an error occurs:

Warning  FailedCreatePodSandBox  3s    kubelet            Failed to create pod sandbox: rpc error: code = NotFound desc = failed to create containerd container: error unpacking image: failed to extract layer sha256:1021ef88c7974bfff89c5a0ec4fd3160daac6c48a075f74cff721f85dd104e68: failed to get reader from content store: content digest sha256:fbe1a72f5dcd08ba4ca3ce3468c742786c1f6578c1f6bb401be1c4620d6ff705: not found

If I remove the image before start the nydus snapshotter, it would be fine.

The test pod yaml in k8s:

apiVersion: v1
kind: Pod
metadata:
  name: test
  namespace: default
spec:
  runtimeClassName: kata
  containers:
  - name: test
    image: quay.io/kata-containers/confidential-containers:unsigned

[ChengyuZhu6]

/cc @jiangliu

[jiangliu]

This is expected to fail:)