search

nebari-dev / nebari

🪴 Nebari - your open source data science platform
https://nebari.dev
BSD 3-Clause "New" or "Revised" License
267 stars 88 forks source link

[ENH] - Add container registries for other cloud providers #1125

Open iameskild opened 2 years ago

iameskild commented 2 years ago

Feature description

Currently QHub can be configured to pull images from a private ECR (regardless of what cloud provider QHu is deployed on). It might make sense to enable users to pull images from other specific cloud registries such as GCR, ACR and the equivalent for Digital Ocean.

Ultimately, these options should be configurable in the qhub-config.yaml.

Value and/or benefit

This allows QHub users who want to use private container registries to do so all from a single cloud provider.

Anything else?

No response

trallard commented 2 years ago

AFAIK this should be readily avaiable - ultimately the only blocker to achieve this should be getting the correct permission level and credentials within QHub.

We can then work on adding a detailed documentation section as the one in https://binderhub.readthedocs.io/en/latest/zero-to-binderhub/setup-registry.html

iameskild commented 2 years ago

Thanks for the link @trallard! If that's the case, we would also want to add these options to the qhub-config schema (just update issue details with that note).

iameskild commented 2 years ago

Currently, the external ECR is created via Terraform. Perhaps we would want to do the same for the others?

danlester commented 2 years ago

I think there are a couple of things in play here:

  1. external_container_reg in the YAML file allows images to be loaded from an external container registry (currently working with AWS ECR) given AWS access key/secret (also specified in the YAML file). This was designed for the distribution of images that are controlled by Quansight, e.g. to be distributed to multiple EQHubs.
  2. The 'local' container registries built by QHub in stage 02-infrastructure. These were designed (I think) for admins of a QHub to have storage for their own versions of the QHub images. If I've understood, this shouldn't need external credentials as it is all built within QHub. So maybe credentials can be passed easily within Terraform, if they are even needed within the same cloud. I'm not sure this functionality is in use by any clients at the moment, or if it is fully functional in the latest codebase.

@iameskild I was talking about (1) above earlier today, but maybe (2) is what you are really interested in. However, you are incorrect in your link above - the external ECR (type 1) is not created by QHub - that code is only for the configuration and management of credentials within the cluster after deployment.

Anyway, for (1) I'm sure other clouds' container registries could be used in place of AWS ECR, and indeed the main difference would be in managing credentials - but in fact that is the bulk of the work here (see Eskild's link above).

costrouc commented 2 years ago

Also there is a blocker in that not all cloud providers behave the same with registries. Digital Ocean if I last remember only allows one registry per account ... not sure if that has changed.

iameskild commented 2 years ago

Thanks for the follow up @danlester and @costrouc! It sounds like we might just need to do some research to determine which of these other registries will need additional dev work. Otherwise we can just use the ECR terraform code as a template. This is not a priority at the moment, I am just tracking some of the requests coming from OT.

rsignell commented 4 months ago

Does this enhancement issue address specification of a custom container registry in the nebari config, as discussed here? https://github.com/orgs/nebari-dev/discussions/2097 If not, I'd be happy to raise a new issue.