Closed jmervine closed 2 years ago
@jmervine Thanks for pointing it out. I met this strange error today when doing testing. I'm now using the same v2.9.3 version plugin against a new Kubernetes v1.21.
Not pretty sure if this is caused by the K8s version or the plugin version. IIRC, the same plugin version with the same config file worked previously.
Please post your configuration. My expectation would this should only be an issue if you define a custom value for tag_to_kubernetes_name_regexp
which could be better documented
Labeling a bug as we can resolve to not require a prefix in the regex
@jcantrill @jmervine This is my configurations for your reference:
<filter event.pod.**.*>
@type kubernetes_metadata
# Use a custom pattern to extract the pod metadata for Kubernetes API lookup.
# The container_name and docker_id are used as a caching key for pod metadata
# so we populate them with the pod UID. Note: pod name may contain '.' character
tag_to_kubernetes_name_regexp 'event\.pod.[^\.]+\.(?<namespace>[^_]+)_(?<pod_name>[^_]+)_(?<container_name>[^_]+)_(?<docker_id>[^_]+)$'
skip_container_metadata true
skip_namespace_metadata true
annotation_match [
"execution_date"
]
</filter>
# Events now have a new "kubernetes" field with the pod labels and annotations
I took a look at the PR but is not very clear of the root cause. Actually, I didn't quite understand how the "tag_to_kubernetes_name_regex" really works.
Could you please share your suggestion to avoid such an IndexError error? Use the old version like v2.9.2 or change the value of tag_to_kubernetes_name_regexp
?
@jcantrill @jmervine This is my configurations for your reference:
<filter event.pod.**.*> @type kubernetes_metadata # Use a custom pattern to extract the pod metadata for Kubernetes API lookup. # The container_name and docker_id are used as a caching key for pod metadata # so we populate them with the pod UID. Note: pod name may contain '.' character tag_to_kubernetes_name_regexp 'event\.pod.[^\.]+\.(?<namespace>[^_]+)_(?<pod_name>[^_]+)_(?<container_name>[^_]+)_(?<docker_id>[^_]+)$' skip_container_metadata true skip_namespace_metadata true annotation_match [ "execution_date" ] </filter> # Events now have a new "kubernetes" field with the pod labels and annotations
I took a look at the PR but is not very clear of the root cause. Actually, I didn't quite understand how the "tag_to_kubernetes_name_regex" really works. Could you please share your suggestion to avoid such an IndexError error? Use the old version like v2.9.2 or change the value of
tag_to_kubernetes_name_regexp
?
This configuration is only required if the running collector is reading logs from a non-standard kubernetes location, something other then /var/log/containers
or /var/log/pods
. This configuration allows the plugin to query the API server to retrieve meta data which requires: namespace, podname. If you mount the default locations into your container then you can remove this config.
Seeing the following
It appears to be related to a (somewhat) hidden change in version 2.9.3, which oddly has no tag. If you look at: https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter/compare/v2.9.2..798febb2c079ecce84348ee593ae6e77b30d1be0#diff-0b4b9044f59b6776e7937e8f714cb59db8858c728efb76dd18456f026f423f19R347
I'm not sure if this is intentional and I'm missing something on my end, but figured it was worth mentioning.