Nephio is a Kubernetes-based automation platform for deploying and managing highly distributed, interconnected workloads such as 5G Network Functions, and the underlying infrastructure on which those workloads depend.
Apache License 2.0
93
stars
52
forks
source link
Figure out how to represent per-environment customizations, for dev and prod #703
Original issue URL: https://github.com/kptdev/kpt/issues/3280
Original issue user: https://github.com/mortent
Original issue created at: 2022-05-27T23:09:17Z
Original issue last updated at: 2023-01-17T06:29:49Z
Original issue body: Keeping in mind blast radius and permissions
Original issue comments:
Comment user: https://github.com/delve
Comment created at: 2022-06-30T15:10:21Z
Comment last updated at: 2022-06-30T15:10:21Z
Comment body: So I have a functioning POC separating env config from cross-env config that looks roughly like
with relevant function configurations as described in the pipeline. This lives in a repo with environments isolated by directory. The purpose being to enable automated promotion without clobbering environment specific details.
I also wonder if it's possible to put the pipeline itself into its own resource to enable something like:
This would allow, for example, application of stricter policy validation in a prod environment. I know we could edit the package in place in prod to do that, but that makes promotion tricky from lower environments.
Comment user: https://github.com/bgrant0607
Comment created at: 2022-06-30T15:25:12Z
Comment last updated at: 2022-06-30T15:25:12Z
Comment body: Thanks for the example, @delve!
Each variant requires variant-specific inputs. If the image tag or digest is one of those inputs, how did you want to propagate a new value across environments?
Comment user: https://github.com/delve
Comment created at: 2022-06-30T15:52:59Z
Comment last updated at: 2022-06-30T15:53:53Z
Comment body: I'm expecting CI automation to copy promotable changes across environment directories. In the overly-simplified example above I should have had image.yaml in a different subdirectory. Something like ./promotable-configs maybe.
Once promotion is approved (either via automated testing or a PR process) the CI copies ./dev/promotable-configs to ./prod/promotable-configs.yaml. And then gitops takes over pushes the change to the cluster.
Comment user: https://github.com/bgrant0607
Comment created at: 2022-06-30T16:33:24Z
Comment last updated at: 2022-06-30T16:33:24Z
Comment body: @delve What do you think about the approach of creating a package with all common attributes and the function pipeline (a "blueprint"), clone one copy of it per environment via kpt pkg get, and change environment-specific attributes in each copy (aka variant)? In order to promote an image tag/digest, it would be changed in the blueprint, and then kpt pkg update would be invoked on each variant.
We're trying to make those types of updates easier with the package orchestrator. An example of a bulk upgrade flow can be seen in the prototype demo: https://www.youtube.com/watch?v=d_iV22_6nAM
Comment user: https://github.com/delve
Comment created at: 2022-07-08T12:33:29Z
Comment last updated at: 2022-07-08T12:33:29Z
Comment body: @bgrant0607 I'm not opposed to that approach, but the org I'm working for is approaching blueprints as generic building blocks agnostic of the deployed code. Instead of 10 blueprints, one per app, we're looking at 3 blueprints, each a composable part, where the variant supplies app details like which image to use.
Under that arrangement it's necessary to maintain configuration data separate from the package. I'm working on implementing a wrapper pattern for this.
Comment user: https://github.com/bgrant0607
Comment created at: 2022-07-08T16:58:52Z
Comment last updated at: 2022-07-08T16:58:52Z
Comment body: @delve I expect kpt to work with multiple levels of derivation, such as generic blueprint -> multi-environment app blueprint -> specific environment deployment package
Comment user: https://github.com/bgrant0607
Comment created at: 2022-07-30T00:35:48Z
Comment last updated at: 2022-07-30T00:35:48Z
Comment body: We'll want a way to be able to set properties in common per environment across multiple applications -- mixins effectively.
Comment user: https://github.com/leeren
Comment created at: 2023-01-17T06:29:49Z
Comment last updated at: 2023-01-17T06:29:49Z
Comment body: Has any progress been made on making propagating variant-specific inputs to generic blueprints with kpt easier? @delve curious what you ended up doing.
I agree that isolating environment-specific config settings by folder and having CI selectively run them seems like the way to go. Is it possible for the kptfile pipeline itself to be templatable to make this easier?
Original issue URL: https://github.com/kptdev/kpt/issues/3280 Original issue user: https://github.com/mortent Original issue created at: 2022-05-27T23:09:17Z Original issue last updated at: 2023-01-17T06:29:49Z Original issue body: Keeping in mind blast radius and permissions
Original issue comments: Comment user: https://github.com/delve Comment created at: 2022-06-30T15:10:21Z Comment last updated at: 2022-06-30T15:10:21Z Comment body: So I have a functioning POC separating env config from cross-env config that looks roughly like
with relevant function configurations as described in the pipeline. This lives in a repo with environments isolated by directory. The purpose being to enable automated promotion without clobbering environment specific details.
I also wonder if it's possible to put the pipeline itself into its own resource to enable something like:
This would allow, for example, application of stricter policy validation in a prod environment. I know we could edit the package in place in prod to do that, but that makes promotion tricky from lower environments.
Comment user: https://github.com/bgrant0607 Comment created at: 2022-06-30T15:25:12Z Comment last updated at: 2022-06-30T15:25:12Z Comment body: Thanks for the example, @delve!
Each variant requires variant-specific inputs. If the image tag or digest is one of those inputs, how did you want to propagate a new value across environments?
Comment user: https://github.com/delve Comment created at: 2022-06-30T15:52:59Z Comment last updated at: 2022-06-30T15:53:53Z Comment body: I'm expecting CI automation to copy promotable changes across environment directories. In the overly-simplified example above I should have had
image.yamlin a different subdirectory. Something like./promotable-configsmaybe.Once promotion is approved (either via automated testing or a PR process) the CI copies
./dev/promotable-configsto./prod/promotable-configs.yaml. And then gitops takes over pushes the change to the cluster.Comment user: https://github.com/bgrant0607 Comment created at: 2022-06-30T16:33:24Z Comment last updated at: 2022-06-30T16:33:24Z Comment body: @delve What do you think about the approach of creating a package with all common attributes and the function pipeline (a "blueprint"), clone one copy of it per environment via kpt pkg get, and change environment-specific attributes in each copy (aka variant)? In order to promote an image tag/digest, it would be changed in the blueprint, and then kpt pkg update would be invoked on each variant.
We're trying to make those types of updates easier with the package orchestrator. An example of a bulk upgrade flow can be seen in the prototype demo: https://www.youtube.com/watch?v=d_iV22_6nAM
Comment user: https://github.com/delve Comment created at: 2022-07-08T12:33:29Z Comment last updated at: 2022-07-08T12:33:29Z Comment body: @bgrant0607 I'm not opposed to that approach, but the org I'm working for is approaching blueprints as generic building blocks agnostic of the deployed code. Instead of 10 blueprints, one per app, we're looking at 3 blueprints, each a composable part, where the variant supplies app details like which image to use.
Under that arrangement it's necessary to maintain configuration data separate from the package. I'm working on implementing a wrapper pattern for this.
Comment user: https://github.com/bgrant0607 Comment created at: 2022-07-08T16:58:52Z Comment last updated at: 2022-07-08T16:58:52Z Comment body: @delve I expect kpt to work with multiple levels of derivation, such as generic blueprint -> multi-environment app blueprint -> specific environment deployment package
Comment user: https://github.com/bgrant0607 Comment created at: 2022-07-30T00:35:48Z Comment last updated at: 2022-07-30T00:35:48Z Comment body: We'll want a way to be able to set properties in common per environment across multiple applications -- mixins effectively.
Comment user: https://github.com/leeren Comment created at: 2023-01-17T06:29:49Z Comment last updated at: 2023-01-17T06:29:49Z Comment body: Has any progress been made on making propagating variant-specific inputs to generic blueprints with kpt easier? @delve curious what you ended up doing.
I agree that isolating environment-specific config settings by folder and having CI selectively run them seems like the way to go. Is it possible for the kptfile pipeline itself to be templatable to make this easier?