search

kubernetes-sigs / kustomize

Customization of kubernetes YAML configurations
Apache License 2.0
10.9k stars 2.24k forks source link

Replacement inbetween #5616

Open Yingrjimsch opened 6 months ago

Yingrjimsch commented 6 months ago

Eschewed features

What would you like to have added?

To replace inbetween to substrings or chars would be very benificial. The replacement by delimiter with an index works pretty good but in some cases it is not enough because the part to replace is between two different chars.

Why is this needed?

If for example I want to replace the major version of nginx:1.7.9 to nginx:2.7.9 there is no right delimiter for me to use to achieve the task. We have this problem in multiple yaml replacements, where the delimiter replacement is not enough.

The following issue tries to achieve something similar: https://github.com/kubernetes-sigs/kustomize/issues/4555 Another example is we have a URL: http://loki-backend.$(ENVIRONMENT)-myapp-observability.svc.cluster.local:3100$request_uri; where we want to replace dynamically the $(ENVIRONMENT) with sta, pre, pro or other environment abbreviations. For now it is not possible to replace this correctly because it is between the char . and the char -

Can you accomplish the motivating task without this feature, and if so, how?

For now the example cannot be solved because it is between the char . and the char -

What other solutions have you considered?

My suggested solution is to have an additional enddelimiter property in replacement.options which can specify the second char / substring for replacement purpose. This can be done in source and target to be consistent and would have no affect on the current functionallity and a minimal code change

replacements:
- source:
    kind: Deployment
    name: deploy2
    fieldPath: spec.template.spec.containers.0.image
    options:
      delimiter: ':'
  targets:
  - select:
      kind: Deployment
      name: deploy1
    fieldPaths:
    - spec.template.spec.containers.1.image
    options:
      delimiter: ':'
      enddelimiter: '.'

Anything else we should know?

I have created a branch on my fork with an implemented version, due to being new to golang I'm open for clean code or performance optimization suggestion. I also included two basic tests in the replacement_test.go which are called partial string replacement with enddelimiter in target - replace and partial string replacement with enddelimiter in source - replace

Feature ownership

k8s-ci-robot commented 6 months ago

This issue is currently awaiting triage.

SIG CLI takes a lead on issue triage for this repo, but any Kubernetes member can accept issues by applying the triage/accepted label.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
k8s-triage-robot commented 3 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

koba1t commented 2 months ago

@Yingrjimsch Hi, I'm so sorry that I can't reply to you with your feature request for around 4 months. Until recently, I couldn't take time to do big work due to having health issues for around half of a year.

I can start to discuss this now. /assign

koba1t commented 2 months ago

First, I am Sorry to do this after you opened a PR, but I think we can't accept the current interface. I think what you really need is the "multiple delimiter", and enddelimiter is not a suitable word because it really shows that second delimiter field.

I believe someone will soon open a PR that requires adding the "third delimiter" field if I accept your current PR.

Yingrjimsch commented 2 months ago

First, I am Sorry to do this after you opened a PR, but I think we can't accept the current interface. I think what you really need is the "multiple delimiter", and enddelimiter is not a suitable word because it really shows that second delimiter field.

I believe someone will soon open a PR that requires adding the "third delimiter" field if I accept your current PR.

Thank you for the response. So in your opinion it should be able to make more than two delimiter? The reason for enddelimiter was to have a better capability (regex, full text replaxe) without breaking the current delimiter function. In my opinion there is the possibility to change delimiter completely to something more flexible but this leads to a breaking change. As we have experienced to fully reach its potential a single delimiter is not enough, do you think different about this?

FYI: a third delimiter wouldn't make sense im my opinion because with regex you can do more complex find and replace functions

Yingrjimsch commented 2 months ago

@Yingrjimsch Hi, I'm so sorry that I can't reply to you with your feature request for around 4 months. Until recently, I couldn't take time to do big work due to having health issues for around half of a year.

I can start to discuss this now. /assign

No problem I hopebyou are feeling better now. If we can work something out that works for me and you approve it I'd appreciate it greatly

koba1t commented 2 months ago

@Yingrjimsch

FYI: a third delimiter wouldn't make sense im my opinion because with regex you can do more complex find and replace functions

Hmm..... If you don't need more complex operations that use more delimiters, I feel that using the technique of multiple replacements and temporary values can achieve what you want in the current kustomize.

For example, https://github.com/koba1t/kustomize-demo/tree/bce00590ad987efe42cbafe71d288adeb9af1d65/replace_complex_domain

In my memory, the former kustomize maintainer preferred to use this.

Yingrjimsch commented 2 months ago

@Yingrjimsch

FYI: a third delimiter wouldn't make sense im my opinion because with regex you can do more complex find and replace functions

Hmm..... If you don't need more complex operations that use more delimiters, I feel that using the technique of multiple replacements and temporary values can achieve what you want in the current kustomize.

For example, https://github.com/koba1t/kustomize-demo/tree/bce00590ad987efe42cbafe71d288adeb9af1d65/replace_complex_domain

In my memory, the former kustomize maintainer preferred to use this.

So how would you go on and replace the example I made in the first comment of the pull request http://loki-backend.$(ENVIRONMENT)-myapp-observability.svc.cluster.local:3100$request_uri; Here I want to replace the $(ENVIRONMENT) or how would you be able to use a source value that is not necessary inside of a configmap? Or another example would be if you have a specific prefix and suffix but you don't want to replace everything in between but everything including prefix and suffix? These things can all be done by my extension but are missing in the current implementation of replacements.

koba1t commented 2 months ago

@Yingrjimsch

So how would you go on and replace the example I made in the first comment of the pull request http://loki-backend.$(ENVIRONMENT)-myapp-observability.svc.cluster.local:3100$request_uri;

First, make configMap, which contains "$(ENVIRONMENT)-myapp-observability" at value. Second, execute replacements that replace $(ENVIRONMENT) in the above value. (use delimiter: "-" ) Then, that configMap contains an intermediate result that you want, like "develop-myapp-observability", and you can use this value for the next replacements. (use delimiter: "." )

Yingrjimsch commented 2 months ago

@koba1t thanks for your answer. I can see how this would work, in my opinion, it is unnecessary complicated but I feel I can't convince you... if you decide there is a use for my pull request I'd be happy to assist.

rolandkool commented 2 months ago

@koba1t I'm not sure if your approach is sufficiently flexible to deal with all kinds of replacement that the inbetween replacement is trying to solve. For example, how would this multiple replacement strategy work for this:

iam.gke.io/gcp-service-account: CLUSTER_NAME-go2k-sa@PROJECT_NAME.iam.gserviceaccount.com

Replacing CLUSTER_NAME is easy, replacing PROJECT_NAME is not. With the inbetween, it could be done like this:

    options:
      delimiter: '@'
      endDelimiter: '.'
      index: 0
Yingrjimsch commented 2 months ago

@koba1t I'm not sure if your approach is sufficiently flexible to deal with all kinds of replacement that the inbetween replacement is trying to solve. For example, how would this multiple replacement strategy work for this:

iam.gke.io/gcp-service-account: CLUSTER_NAME-go2k-sa@PROJECT_NAME.iam.gserviceaccount.com

Replacing CLUSTER_NAME is easy, replacing PROJECT_NAME is not. With the inbetween, it could be done like this:

    options:
      delimiter: '@'
      endDelimiter: '.'
      index: 0

I agree thanks for the great example even CLUSTER_NAME seems to be not quite easy to replace with the berween replacement you could do

    options:
      delimiter: ' '
      endDelimiter: '-'
      index: 0

out if curiosity how would you do it without?

rolandkool commented 2 months ago

CLUSTER_NAME can be replaced using delimiter set to - and picking index 0. Replacing the other part can't be done at the moment, so our workaround is to replace the whole line with the desired value, which introduces a lot of 'redundant' configuration.

Yingrjimsch commented 2 months ago

@koba1t hi kobalt any don't you think this is still relevant? If not, I've got another issue with the replacements, I want to get labelSelector:env=this|that as written here this should be possible, the label selector should be implemented this way https://github.com/kubernetes-sigs/kustomize/blob/master/api/types/selector.go#L21 can you tell me how I can achieve that?

k8s-triage-robot commented 1 month ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

koba1t commented 2 weeks ago

@Yingrjimsch Sorry, I didn't completely understand what you want.

replacements didn't support the label selector use because replacements replaced any value from one resource to another. That doesn't support any selector that may be multiple resources returned. https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/replacements/