search

kubernetes-sigs / kustomize

Customization of kubernetes YAML configurations
Apache License 2.0
10.98k stars 2.25k forks source link

improper change of block scalar chomp in config generator #886

Closed donbowman closed 4 years ago

donbowman commented 5 years ago

I have a simple kustomization.yaml file

configMapGenerator:
  - name: env-config
    literals:
      - >-
        druid_extensions_loadList=["druid-histogram",
          "druid-datasketches",
          "druid-lookups-cached-global",
          "druid-azure-extensions",
          "postgresql-metadata-storage",
          "druid-rabbitmq",
          "druid-basic-security",
          "druid-virtual-columns",
          "druid-kafka-indexing-service"]

When build, the output is:

apiVersion: v1
data:
  druid_extensions_loadList: |-
    ["druid-histogram",
      "druid-datasketches",
      "druid-lookups-cached-global",
      "druid-azure-extensions",
      "postgresql-metadata-storage",
      "druid-rabbitmq",
      "druid-basic-security",
      "druid-virtual-columns",
      "druid-kafka-indexing-service"]
kind: ConfigMap
metadata:
  name: env-config-m26mgthb46

which is incorrect, it has been changed from block scalar single newline chomp to block scalar multiline.

since I am using this ulimately as an environment variable, this breaks.

vreon commented 5 years ago

Do I understand correctly that you want the resulting string to be free of newlines? If so, the block scalar single newline chomp isn't working in the original document. :thinking:

Here's a test with a Python YAML parser:

>>> import yaml

>>> value_before = yaml.safe_load(
"""literals:
  - >-
    druid_extensions_loadList=["druid-histogram",
      "druid-datasketches",
      "druid-lookups-cached-global",
      "druid-azure-extensions",
      "postgresql-metadata-storage",
      "druid-rabbitmq",
      "druid-basic-security",
      "druid-virtual-columns",
      "druid-kafka-indexing-service"]
""")['literals'][0]

>>> # Chop off the key and equals so that we see the value that Kustomize sees
>>> value_before = value_before[len('druid_extensions_loadList='):]

>>> '\n' in value_before
True

Indeed, the string that Kustomize produces is identical to the original one:

>>> value_after = yaml.safe_load(
"""druid_extensions_loadList: |-
  ["druid-histogram",
    "druid-datasketches",
    "druid-lookups-cached-global",
    "druid-azure-extensions",
    "postgresql-metadata-storage",
    "druid-rabbitmq",
    "druid-basic-security",
    "druid-virtual-columns",
    "druid-kafka-indexing-service"]
""")['druid_extensions_loadList']

>>> value_before == value_after
True

I'm not sure what the correct YAML syntax here is if you want to use a block scalar, but after tinkering with this online parser I found that a flow scalar happens to work:

>>> value_oneline = yaml.safe_load(
"""literals:
  - druid_extensions_loadList=["druid-histogram",
      "druid-datasketches",
      "druid-lookups-cached-global",
      "druid-azure-extensions",
      "postgresql-metadata-storage",
      "druid-rabbitmq",
      "druid-basic-security",
      "druid-virtual-columns",
      "druid-kafka-indexing-service"]
""")['literals'][0][len('druid_extensions_loadList='):]

>>> '\n' in value_oneline
False

>>> print(type(value_oneline))
<class 'str'>

>>> print(value_oneline)
["druid-histogram", "druid-datasketches", "druid-lookups-cached-global", "druid-azure-extensions", "postgresql-metadata-storage", "druid-rabbitmq", "druid-basic-security", "druid-virtual-columns", "druid-kafka-indexing-service"]
donbowman commented 5 years ago

Thanks. Switching to this syntax was to work around https://github.com/kubernetes-sigs/kustomize/issues/843

I can try the flow scalar again i suppose

fejta-bot commented 5 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

fejta-bot commented 5 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle rotten

donbowman commented 5 years ago

/remove-lifecycle rotten

its really a shame the only thing visiting the issues is the bot.

the quoting is really tough to get right in kustomize, navigating its buglist.

fejta-bot commented 5 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

fejta-bot commented 4 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle rotten

fejta-bot commented 4 years ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /close

k8s-ci-robot commented 4 years ago

@fejta-bot: Closing this issue.

In response to [this](https://github.com/kubernetes-sigs/kustomize/issues/886#issuecomment-565498698): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >Send feedback to sig-testing, kubernetes/test-infra and/or [fejta](https://github.com/fejta). >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.