search

rubrikinc / fluent-plugin-throttle

Rate limiting for fluentd
Apache License 2.0
41 stars 24 forks source link

multiple groups in group_key resolving to `nil` #6

Closed george-angel closed 5 years ago

george-angel commented 5 years ago

Using namespace key group_key "kube_namespace_name":

fluentd-2 fluentd 2018-11-23 10:43:01 +0000 [warn]: rate exceeded group_key="kube-system" rate_s=3 period_s=60 limit=10 rate_limit_s=0 reset_rate_s=0
fluentd-2 fluentd 2018-11-23 10:43:02 +0000 [warn]: rate exceeded group_key="sys-mon" rate_s=1 period_s=60 limit=10 rate_limit_s=0 reset_rate_s=0
fluentd-0 fluentd 2018-11-23 10:42:45 +0000 [warn]: rate exceeded group_key="kube-system" rate_s=3 period_s=60 limit=10 rate_limit_s=0 reset_rate_s=0
fluentd-0 fluentd 2018-11-23 10:42:47 +0000 [warn]: rate exceeded group_key="telecom" rate_s=0 period_s=60 limit=10 rate_limit_s=0 reset_rate_s=0
fluentd-0 fluentd 2018-11-23 10:42:52 +0000 [warn]: rate exceeded group_key="sys-mon" rate_s=0 period_s=60 limit=10 rate_limit_s=0 reset_rate_s=0

using kube_pod_name key group_key "kube_pod_name" :

fluentd-0 fluentd 2018-11-23 10:38:12 +0000 [warn]: rate exceeded group_key="grafana-84dfb4866b-zpdtx" rate_s=0 period_s=60 limit=10 rate_limit_s=0 reset_rate_s=0
fluentd-0 fluentd 2018-11-23 10:38:14 +0000 [warn]: rate exceeded group_key="blackbox-exporter-7f75d97d67-z9xtc" rate_s=0 period_s=60 limit=10 rate_limit_s=0 reset_rate_s=0
fluentd-0 fluentd 2018-11-23 10:38:14 +0000 [warn]: rate exceeded group_key="kube-apiserver-ip-10-66-23-164.eu-west-1.compute.internal" rate_s=1 period_s=60 limit=10 rate_limit_s=0 reset_rate_s=0
fluentd-2 fluentd 2018-11-23 10:38:06 +0000 [warn]: rate exceeded group_key="kube-apiserver-ip-10-66-23-164.eu-west-1.compute.internal" rate_s=1 period_s=60 limit=10 rate_limit_s=0 reset_rate_s=0
fluentd-2 fluentd 2018-11-23 10:38:09 +0000 [warn]: rate exceeded group_key="certificate-authority-77ff9b9fbc-n72m8" rate_s=0 period_s=60 limit=10 rate_limit_s=0 reset_rate_s=0

Using both: group_key "kube_namespace_name,kube_pod_name" :

fluentd-0 fluentd 2018-11-23 10:44:24 +0000 [warn]: rate exceeded group_key=nil rate_s=29 period_s=60 limit=10 rate_limit_s=0 reset_rate_s=0
fluentd-2 fluentd 2018-11-23 10:44:27 +0000 [warn]: rate exceeded group_key=nil rate_s=5 period_s=60 limit=10 rate_limit_s=0 reset_rate_s=0
fluentd-0 fluentd 2018-11-23 10:44:35 +0000 [warn]: rate exceeded group_key=nil rate_s=1 period_s=60 limit=10 rate_limit_s=0 reset_rate_s=0

example message:

{
  "date": 1542973504.01747,
  "log": "I1123 11:45:04.017354       1 logs.go:49] http: TLS handshake error from 10.66.24.95:29201: EOF\n",
  "stream": "stderr",
  "kube_pod_name": "kube-apiserver-ip-10-66-23-81.eu-west-1.compute.internal",
  "kube_namespace_name": "kube-system",
  "kube_pod_id": "f33508e4-dcf6-11e8-9adf-060b9e31fc68",
  "kube_labels": {
    "k8s-app": "apiserver"
  },
  "kube_host": "ip-10-66-23-81.eu-west-1.compute.internal",
  "kube_container_name": "kube-apiserver",
  "kube_docker_id": "e7b3eea81c35c558defcd2418a7953bab7932a33c90981dccdc73c198f730007",
  "origin": "exp-1-aws"
}
george-angel commented 5 years ago

Aha, I'm using the latest available gem https://rubygems.org/gems/fluent-plugin-throttle/versions/0.0.3 - which doesn't have composite keys.

Would be mind creating a new release with latest changes please? tyvm

shaikatz commented 5 years ago

@rfitzhugh can you please publish your latest code to rubygems repo?

bombela commented 5 years ago

https://rubygems.org/gems/fluent-plugin-throttle/versions/0.0.4