search

kyma-project / kyma

Kyma is an opinionated set of Kubernetes-based modular building blocks, including all necessary capabilities to develop and run enterprise-grade cloud-native applications.
https://kyma-project.io
Apache License 2.0
1.51k stars 406 forks source link

Open internal docker registry for customers #18555

Closed pbochynski closed 2 months ago

pbochynski commented 10 months ago

Description Provide a way to use in-cluster docker registry for hosting customer's images. As the registry is accessible with the node-port it should be possible to push the images from the developer's machine or from CI/CD agent using port forwarding. Provide a command in the kyma CLI to push images.

AC

Reasons Pushing images directly to the in-cluster registry simplifies the development, testing and even production releases. Users should be able to easily deploy their code and run their tests without the need for additional BTP or hyperscaler services. Currently a problem arises when users want to deploy their code to the cluster, due to the fact that there is no SAP registry available. Unlike with CF where a cf push does all the work necessary to deploy the code.

Related Issues Enable Registry Cache Extension

kwiatekus commented 8 months ago

Allow serverless module users to configure custom HA store as serverless docker registry backend.

kwiatekus commented 8 months ago

@pbochynski we see more and more reasons to extract docker registry into separate module. Any objections? We would like to try it out and do it in a way that serverless (untill its still dependant on internal docker registry) ux doesnt suffer https://github.com/kyma-project/docker-registry/issues/19

pbochynski commented 8 months ago

@pbochynski we see more and more reasons to extract docker registry into separate module. Any objections?

The only problem I see is that the serverless module will not work out of the box. You would introduce the dependency to registry module or to external registry. But maybe it is another reason to consider changing serverless module in the way that it doesn't have to build a container. Just load function code from CR and cache dependencies on persistent volume. What do you think?

kwiatekus commented 8 months ago

Thanks @pbochynski . That would mean that serverless runs pre-built containers as function runtimes with code dependencies mounted. dependencies resolving would need to be moved from built time into runtime ( start time ). There is a consequence in function scaling scenario: i.e when a new replica starts it would need to resolve dependencies ( i.e npm install ) into a mounted volume. In theory, replicas could have different versions of dependencies. But if we consider serverless being dev module only that should be ok.

kwiatekus commented 8 months ago

Opening internal docker registry should be considered as extending toolset for developers to run/test their applications on kyma only in develoment mode. We should keep recommending full-fledged HA external docker registry for production grade installations of customer applications.

kwiatekus commented 7 months ago

We aim to separate docker registry reconciliation to a dedicated operator, which could be used both in and out of sereverles context.

kwiatekus commented 7 months ago

The team is also looking into using the latest version of distribution (a.k.a registry) from cncf ->

pPrecel commented 6 months ago

@pbochynski we see more and more reasons to extract docker registry into separate module. Any objections?

The only problem I see is that the serverless module will not work out of the box. You would introduce the dependency to registry module or to external registry. But maybe it is another reason to consider changing serverless module in the way that it doesn't have to build a container. Just load function code from CR and cache dependencies on persistent volume. What do you think?

I think, in this case, it would be nice to have one PVC for all functions, not to reserve space for every function. The problem is that almost all types of PVC are namespace-scoped so it would be impossible to use simple PVC. The only solution I see right now is to use nfs (network-file-system) that allows to share volume between namespaces. It's something to investigate

kwiatekus commented 6 months ago

Following recent feedback we will look into allowing using the module via docker CLI https://github.com/kyma-project/docker-registry/issues/38

kwiatekus commented 5 months ago

Currently working on exposing docker regisrty via https ( https://github.com/kyma-project/docker-registry/issues/37) and configuring s3-compliant backend for storage (https://github.com/kyma-project/docker-registry/issues/25)

ETA On track

kwiatekus commented 5 months ago

We work on allowing setting up AWS or Azure storage for docker registry. Next will be allowing GCS buckets & allowing configuring BTP secret as a Object Storage. We are finalising exposing docker registry via https.

ETA on track

kwiatekus commented 4 months ago

Status update 15.07.2024

Next plans:

kwiatekus commented 4 months ago

Status update 29.07.2024

kwiatekus commented 3 months ago

Status update 26.08.2024

@kyma-project/otters are working on feature completeness:

Additionally, from the DoD we are working on

We estimate we need extra week to finish New forecast: 13.09.2024

kwiatekus commented 2 months ago

Status update 09.09.2024

kwiatekus commented 2 months ago

The docker registry is available for internal customers on canary landscape (via experimental channel ).

What's new announcement Internal docs