Open rudyeeee opened 8 years ago
Could you provide a little more detail on the code you're running? Do you have a minimum viable program that exhibits this?
If you're using PacketSource or ReadPacketData, maybe switch to ReadPacketDataTo to avoid an extra alloc. Doesn't look like we've implemented ZeroCopyReadPacketData for pfring, but we probably should.
Are you doing other stuff in your code, tcp assembly or the like?
Could you generate a CPU profile with your run (https://golang.org/pkg/runtime/pprof/) and generate a SVG of its output to add here? That might point to a smoking gun.
I have a similar issue with PF_RING. Specifically the latencies seem very high.
I have created a minimal ping
using gopacket and pfring here: https://github.com/redsift/pingdemo
When testing this on Ubuntu 16.04.1 LTS with PF_RING 6.6.0 and the pfring e1000e driver, the latency suggests something is wrong.
The linked project fires a ping packet and times the response.
Hitting 8.8.8.8 takes ~500ms
PF_RING ping timing demo: ping 8.8.8.8 from 10.60.0.175
ping time=500.586337ms
ping time=500.571822ms
ping time=500.571827ms
ping time=500.575797ms
ping time=500.577616ms
The same test with the zero copy interface is mostly the same though I guess this is not relevant for my single packet test case.
PF_RING ping timing demo: ping 8.8.8.8 from 10.60.0.175
#########################################################################
# ERROR: You do not seem to have a valid PF_RING ZC license 6.6.0.170424 for eno1 [Intel 1 Gbit e1000e family]
# ERROR: Please get one at http://shop.ntop.org/.
#########################################################################
# We're now working in demo mode with packet capture and
# transmission limited to 5 minutes
#########################################################################
ping time=500.57761ms
ping time=500.579852ms
ping time=500.565223ms
ping time=500.56318ms
ping time=500.569313ms
As a control, command line ping reports tiny latencies.
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=59 time=4.50 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=59 time=4.12 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=59 time=4.07 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=59 time=4.41 ms
64 bytes from 8.8.8.8: icmp_seq=5 ttl=59 time=4.20 ms
In another application, gopacket with libpcap reports times within 10% of the command line ping to this ~500ms latency is specific to PF_RING.
What am I missing?
Not sure if this is correct, but my guess is that PF_RING might be trying to buffer multiple packets before returning the set to you, the caller. If you're only seeing one, it may be waiting for a timeout (500ms) before returning. I'll dig briefly into the code and see if that makes sense.
Hmm, there's a very suspicious pfring global called DEFAULT_POLL_DURATION set to 500.
We might need to expose pfring_set_poll_duration to fix this...
Could you throw some printf-debugging around the ReadPacketDataTo call and make sure it's the function that's actually taking ~500ms?
Yes, the 500ms is in ReadPacketDataTo.
Looks like you are right. Combination of pfring_set_poll_duration
(default = 500) and pfring_set_poll_watermark
(default = 128). Clients of gopacket likely want to tweak these based on use case.
I suspect the original issue here is not what I was reporting but looks like https://github.com/google/gopacket/issues/185 clears my problem.
I have added SetPollDuration, SetPollWatermark APIs for performance tweaking needs. Pulled to master recently.
PF_RING NOT FREE. I got same problem . i use dpdk at last.
Same problem here, in input flow. Sending a UDP packet with netcat, netcat exits almost immediately but I see the packet only 2 or 3 seconds later. The ping to the remote is 20-30 ms
regardless of the number of slots, Using the pfring in 1gbps occured a lot of drop(Tot pkt Lost)