google / gopacket

Provides packet processing capabilities for Go
BSD 3-Clause "New" or "Revised" License
6.32k stars 1.13k forks source link

How to access the af_packet statistics? #459

Closed styleex closed 5 years ago

styleex commented 6 years ago

Type SocketStatsV3 is a structure from C, how can I access the values of its fields?

The following code is not compiled:

s1, s2, err: = inputAFpacket.SocketStats ()
log.Println (s2.tp_drops)
gconnell commented 6 years ago

Hey, I don't know specifically, but try this first:

log.Printf("%+v", s2)

See what that shows, and it might help out. This is a general solution, so if it works here it'll work anywhere :D

styleex commented 6 years ago

Yes, it works :-D But I would like to export these values to prometheus

gconnell commented 6 years ago

What does it print? I'm hoping that if it prints foo: 1, then s2.foo will work.

styleex commented 6 years ago

Through the reflect package, we can access the unexported structure data. This is what fmt.Printf does

gconnell commented 6 years ago

Hmm, I really thought that s2.tp_drops would work... can you paste the compile error you're getting to this thread, as well as the output of the %+v printf?

styleex commented 6 years ago

Yes.

log.Printf("%+v", s2) prints {tp_packets:273306075 tp_drops:92749 tp_freeze_q_cnt:11}

s2.tp_drops build error: 2.tp_drops undefined (cannot refer to unexported field or method tp_drops)

gconnell commented 6 years ago

Well that's weird and annoying. Guess we'll have to add some methods or export a new struct to make that usable.

styleex commented 6 years ago

By the way, why does summing up the received statistics with the old state, and not overwriting the old values? https://github.com/google/gopacket/blob/master/afpacket/afpacket.go#L348

gconnell commented 6 years ago

To answer your last question, it's because every getsockopt call resets the counters, and we wanted to provide the counters since the beginning of time.

styleex commented 6 years ago

By the way, for exporting metrics, for example, in prometheus it would be more convenient if the difference between two readings was given. Then you can use the metric type Counter, not Guage.

In my opinion, generally, the binding with the code should be as transparent as possible and do not make unnecessary logic. Preaggregation of the metric can be done in a higher-level code.

gconnell commented 6 years ago

Use of tools like prometheus is actually exactly why we sum. Counter assumes the value never resets to zero and is from beginning-of-time. The problem with resetting to zero is that the sample rate matters a ton, while if we DONT reset to zero, we can accidentally miss a sampling and still be fine.

styleex commented 6 years ago

Well, not exactly. Counter is stored on the monitoring system side, we send it diffs of this metric and the monitoring system itself adds this diff with the old value.

I mean that this summation can be performed anywhere, and not in binding:

  1. In a high-level code (golang instead of cgo)
  2. In the monitoring system

Therefore, it would be cool if the method of obtaining statistics worked exactly as described in the manuals. And another, higher-level code-utility for golang, could preaggregate it and give pre-aggregated values.

gconnell commented 6 years ago

Oh, you want to use https://godoc.org/github.com/prometheus/client_golang/prometheus#CounterFunc, not https://godoc.org/github.com/prometheus/client_golang/prometheus#Counter and you'll be fine.

notti commented 5 years ago

@styleex: was this resolved with #463 and the suggestions by gconnel?

notti commented 5 years ago

Closing this for now, since it looks resolved.