[Bug]: `extra_tags` on kube_karpenter_node_pools can cause the provisioner to error

[mschnee]

Prior Search

• [X] I have already searched this project's issues to determine if a bug report has already been made.

What happened?

Because extra_tags is used by kube_labels and feeds into matching_labels, applying extra tags can cause the node pool definitions to go over the limit of the 30-requirement limit karpenter has.

Steps to Reproduce

Use a handful of extra tags in kube_karpenter_node_pools at any level (module, region, envirionment, global)

Version

main (development branch)

Relevant log output

"creating node claim, NodeClaim.karpenter.sh \"burstable-ws8kc\" is invalid: [spec.requirements: Too many: 33: must have at most 30 items, <nil>: Invalid value: \"null\": some validation rules were not checked because the object was invalid; correct the existing errors to complete validation]; ...

Recommendations

Just some ideas:

• have separate extra_tags and extra_labels configurations.
• Support removing extra_tags at a lower level (module.yaml) that had at a higher level?

  extra_tags:
  my_tag_to_remove: "__remove__"
  extra_labels:
  my_label_to_remove: "__remove__"

• Finally, given the strictness, perhaps using kube_labels in this manner is inappropriate for this module.

There are a number of issues with using the extra_tags configuration for both AWS Resource Tagging as well as kubernetes labels:

• k8s label names have extremely strict requirements
• k8s label values also have
• extra_tags may end up in unexpected places (such as in the matching labels)
• We may want to explicitly define different naming/value conventions, e.g. java-ish domain notation for AWS tags (net:corgeek:webapp=admin) versus k8s (corgeek.net/webapp=admin)

[fullykubed]

@mschnee I have added 5 extra_tags to the reference architecture and added a bunch of invalid characters.

In the latest commit, I believe that all parts of the stack can now function normally with arbitrary amounts of extra tags with arbitrary characters.

I am going to close this issue, but if you find you are still running into problems with tags, please re-open it, and we can continue to investigate.