Feature: Kubernetes Pods Crawling

[tatsuhirochiba]

Description

This proposal enables crawler user to collect Kubernetes managed Pod metrics. In Kubernetes, one or more containers are grouped into a Pod, so this feature collects/materializes/emits the metrics per Pod.

Here is a list for all related PRs and TODO items for PRs.

• [ ] Design: k8s pods crawling interface, functions and reference implementation (PR #246)
• [ ] Implementation: emitter format for k8s crawler
• [ ] Implementation: support all plugins for k8s crawler
• [ ] Others: Crawler runs as k8s daemon

How to Reproduce

Here is an example command to get redis pod metrics on Kuberenetes. The pod and crawler run on the same machine. python crawler/crawler.py --feature redis --crawlmode K8S --frequency 10 --numprocess 1 --url file:///tmp/crawler/log/t --format csv

Log Output

[nadgowdas]

@tatsuhirochiba We have a manifest to deploy crawler as a deamonset