Closed jerroydmoore closed 7 years ago
Thank you for the contribution @jerroydmoore! Kubernetes support is something we definitely want to improve, and these contributions make our work easier!
As this is not exactly an issue with current images I'm closing this thread, most likely this kind of conversation fits better in Beats discuss forum, but as I said: thank you so much, contributions and experiences are really welcomed :+1:
@jerroydmoore Excellent, thank you! I have pods running in different namespaces. Do you happen to know if it's possible to somehow capture the k8s namespace and send it to kibana with the rest of the data from packetbeat?
Hi @manuel-palacio I'm glad you found it useful! Unfortunately, the packet would have to hook into kubernetes API. Since I have limited knowledge in golang or kubernetes, (and I have higher priority work than adding this metadata), I have not looked into it further. Kubernetes does have an extensive API. The fluentd agent kubernetes ships out of the box has kubernetes metadata attached to the record before being sent to elasticsearch. Maybe you can borrow from this metadata plugin.
Hey Elastic!
I wasn't sure where a good place for this was, but I thought other users might benefit from how I setup packetbeat in k8s.
Here is my daemonset configuration. Note that using a daemonset instead of a Pod ensures that each physical host gets a packetbeat container
One of the things I've done for metricbeat is create an put my ElasticSearch endpoint in an ExternalName Service, so that in my metricbeat.yml, I can point my elasticsearch destination to this dns name rather than the actual dns name. This is the preferred pattern for routing network traffic to destinations outside your k8s cluster. This works great for metricbeat, however, breaks for packetbeat. I suspect, because packetbeat requires
hostNetwork: true
, it bypasseskube-dns
, which holds k8s service information. The work around is to point packetbeat directly to your elasticsearch, not using k8s services. Here is my k8s service for reference if someone is able to figure out how to use k8s services on the host network, that would be fantastic.Then in your metricbeat.yml/packetbeat.yml configuration