search

kubevirt / containerized-data-importer

Data Import Service for kubernetes, designed with kubevirt in mind.
Apache License 2.0
400 stars 254 forks source link

CDI Pulling Images from ECR #1745

Closed MarkSpencerTan closed 2 years ago

MarkSpencerTan commented 3 years ago

Is there any way at the moment to be able to pull disk images from AWS ECR? I know the disk registry option allows a secret to be passed containing the secret/token credential of the repo, however, with ECR, the tokens are limited to a few hours until they refresh so I was wondering if we can somehow pull without needing the credentials.

Our AWS team told us that CDI should just be able to pull images using an IAM role that allows it to pull from the repo. Usually you can link an IAM Role using annotations to a service account being used by the service (CDI). I did notice that CDI has a couple of different service accounts so I'm not too sure which one to use.

Would this be doable right now with CDI or does it have to be implemented? Thank you

/kind enhancement

What happened: Tried to specify a repo from AWS ECR to pull disk images from The Image Pull failed due to error:


I0408 18:30:20.447651       1 importer.go:52] Starting importer
I0408 18:30:20.450319       1 importer.go:134] begin import process
I0408 18:30:20.450340       1 data-processor.go:356] Calculating available size
I0408 18:30:20.450351       1 data-processor.go:368] Checking out file system volume size.
I0408 18:30:20.451118       1 data-processor.go:376] Request image size not empty.
I0408 18:30:20.451131       1 data-processor.go:381] Target size 15Gi.
I0408 18:30:20.452291       1 data-processor.go:238] New phase: TransferScratch
I0408 18:30:20.452308       1 registry-datasource.go:82] Copying registry image to scratch space.
I0408 18:30:20.452314       1 transport.go:173] Downloading image from 'docker://XXX.dkr.ecr.us-west-2.amazonaws.com/pac-virtualrack-vmidisks:gateway-cloudinit-1.1.0', copying file from 'disk' to '/scratch'
E0408 18:30:20.535107       1 transport.go:75] Could not create image reference: Error reading manifest gateway-cloudinit-1.1.0 in XXX.dkr.ecr.us-west-2.amazonaws.com/pac-virtualrack-vmidisks: unauthorized: authentication required
E0408 18:30:20.535175       1 data-processor.go:235] unauthorized: authentication required
Error reading manifest gateway-cloudinit-1.1.0 in 229688178637.dkr.ecr.us-west-2.amazonaws.com/pac-virtualrack-vmidisks

What you expected to happen: CDI can pull from ECR repo

How to reproduce it (as minimally and precisely as possible): Try to pull an image from an ECR repo

Anything else we need to know?: We're using an EKS cluster with Kubevirt. Trying to create persistent VMs using CDI pulling from ECR repositories

Environment:

kubevirt-bot commented 3 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

kubevirt-bot commented 3 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten

kubevirt-bot commented 2 years ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

/close

kubevirt-bot commented 2 years ago

@kubevirt-bot: Closing this issue.

In response to [this](https://github.com/kubevirt/containerized-data-importer/issues/1745#issuecomment-913218310): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
ianb-mp commented 3 months ago

Pulling from ECR can be done like this:

  1. Create a k8s secret of type kubernetes.io/dockerconfigjson using credentials from Docker/Podman e.g. 

    kubectl create -n <namespace> secret generic ecr-creds --from-file=.dockerconfigjson=$HOME/.docker/config.json --type=kubernetes.io/dockerconfigjson

    (replace $HOME/.docker/config.json with the path to your Docker credentials)

  2. Modify the default service account in your namespace to use this secret when pulling images e.g.

    kubectl patch serviceaccount default -n <namespace> \
-p '{"imagePullSecrets": [{"name": "ecr-creds"}]}'

  3. Ensure that your DataVolume manifest uses pullMethod: node (see docs) and same namespace as above