phaag / nfdump

Netflow processing tools
Other
760 stars 198 forks source link

Palo Alto ICMP type not getting through... #333

Closed ken-adey closed 2 years ago

ken-adey commented 2 years ago

Palo Alto ICMP type not getting through nfapd/nfdump. As seen in the below nfdump output and image of the Wireshark packet/flow, the ICMP type 8 is not in the nfdump output:

Flow Record: Flags = 0x46 EVENT, Unsampled label = export sysid = 8 size = 104 first = 1647459540 [2022-03-16 19:39:00] last = 1647459540 [2022-03-16 19:39:00] msec_first = 0 msec_last = 0 src addr = 10.6.68.7 dst addr = 10.206.202.241 ICMP = 0.0 type.code fwd status = 0 tcp flags = 0x00 ........ biFlow Dir = 0x00 arbitrary end reason = 0x00 proto = 1 ICMP (src)tos = 0 (in)packets = 1 (in)bytes = 46 input = 3 output = 3 src mask = 0 /0 dst mask = 0 /0 dst tos = 0 direction = 0 ip router = 10.11.2.4 engine type = 0 engine ID = 0 received at = 1647459359569 [2022-03-16 19:35:59.569] connect ID = 0 fw event = 1: CREATE fw ext event = 0: Ignore secgroup tag = 0 Event time = 0 [1970-01-01 00:00:00.000]

Screenshot from 2022-03-17 09-48-09

ken-adey commented 2 years ago

Pcap file can be provided.

phaag commented 2 years ago

Yes please - send me the pcap, or a link to download. Find my email address in the Authors file.

phaag commented 2 years ago

Hmm .. looks like everything is decode correctly:

Flow Record:
  Flags        =              0x06 NETFLOW v9, Unsampled
  label        =            <none>
  export sysid =                 3
  size         =               104
  first        =        1647459000 [2022-03-16 20:30:00]
  last         =        1647459000 [2022-03-16 20:30:00]
  msec_first   =                 0
  msec_last    =                 0
  src addr     =        xx.6.34.13
  dst addr     =         xx.6.38.5
  ICMP         =               8.0  type.code
  fwd status   =                 0
  tcp flags    =              0x00 ........
  biFlow Dir   =              0x00 arbitrary
  end reason   =              0x00
  proto        =                 1 ICMP
  (src)tos     =                 0
  (in)packets  =                 1
  (in)bytes    =                74
  input        =                 3
  output       =                 3
  src mask     =                 0 /0
  dst mask     =                 0 /0
  dst tos      =                 0
  direction    =                 0
  ip router    =         10.11.0.4
  engine type  =                 0
  engine ID    =                 0
  received at  =     1648225300967 [2022-03-25 17:21:40.967]

Did you try it with the master branch or another version?

ken-adey commented 2 years ago

Yeah, should've mentioned it was nfcapd: Version: 1.6.23 as of 7 months ago.

I notice that master is still 1.6.23.

On Fri, Mar 25, 2022 at 12:25 PM Peter Haag @.***> wrote:

Hmm .. looks like everything is decode correctly:

Flow Record: Flags = 0x06 NETFLOW v9, Unsampled label = export sysid = 3 size = 104 first = 1647459000 [2022-03-16 20:30:00] last = 1647459000 [2022-03-16 20:30:00] msec_first = 0 msec_last = 0 src addr = xx.6.34.13 dst addr = xx.6.38.5 ICMP = 8.0 type.code fwd status = 0 tcp flags = 0x00 ........ biFlow Dir = 0x00 arbitrary end reason = 0x00 proto = 1 ICMP (src)tos = 0 (in)packets = 1 (in)bytes = 74 input = 3 output = 3 src mask = 0 /0 dst mask = 0 /0 dst tos = 0 direction = 0 ip router = 10.11.0.4 engine type = 0 engine ID = 0 received at = 1648225300967 [2022-03-25 17:21:40.967]

Did you try it with the master branch or another version?

— Reply to this email directly, view it on GitHub https://github.com/phaag/nfdump/issues/333#issuecomment-1079191409, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHUIHYHFH2U62QD44EFNQ23VBXSGTANCNFSM5RHQOEPQ . You are receiving this because you authored the thread.Message ID: @.***>

-- Ken Adey

CyGlass, Inc. is a wholly-owned subsidiary of Nominet UK. Nominet UK is registered in England and Wales No. 3203859

This message is intended exclusively for the individual(s) to whom it is addressed and may contain information that is privileged, or confidential. If you are not the addressee, you must not read, use or disclose the contents of this e-mail. If you receive this e-mail in error, please advise us immediately and delete the e-mail. CyGlass, Inc. has taken every reasonable precaution to ensure that any attachment to this e-mail has been swept for viruses. However, Nominet cannot accept liability for any damage sustained as a result of software viruses and would advise that you carry out your own virus checks before opening any attachment

phaag commented 2 years ago

could you try the master branch to verify that? Maybe the master branch should be tagged more clear - sorry. 1.6.24 should follow soon

ken-adey commented 2 years ago

I'm trying to build master based on "./configure --enable-nsel --enable-sflow --enable-readpcap", but running nfcapd I get:

@.***:~/nfdump$ ./bin/nfcapd -f VA01-diagnostic.pcap -l . Add extension: 2 byte input/output interface index Add extension: 4 byte input/output interface index Add extension: 2 byte src/dst AS number Add extension: 4 byte src/dst AS number Add extension: dst tos, direction, src/dst mask Add extension: IPv4 next hop Add extension: IPv6 next hop Add extension: IPv4 BGP next IP Add extension: IPv6 BGP next IP Add extension: src/dst vlan id Add extension: 4 byte output packets Add extension: 8 byte output packets Add extension: 4 byte output bytes Add extension: 8 byte output bytes Add extension: 4 byte aggregated flows Add extension: 8 byte aggregated flows Add extension: in src/out dst mac address Add extension: in dst/out src mac address Add extension: MPLS Labels Add extension: IPv4 router IP addr Add extension: IPv6 router IP addr Add extension: router ID Add extension: BGP adjacent prev/next AS Add extension: time packet received Add extension: NSEL Common block Add extension: NSEL xlate ports Add extension: NSEL xlate IPv4 addr Add extension: NSEL xlate IPv6 addr Add extension: NSEL ACL ingress/egress acl ID Add extension: NSEL username Add extension: NSEL max username Add extension: nprobe/nfpcapd latency Add extension: NEL Common block Add extension: Compat NEL IPv4 Add extension: NAT Port Block Allocation Setup pcap reader Can't init pcap: Snooping not on an ethernet.

On Fri, Mar 25, 2022 at 2:15 PM Peter Haag @.***> wrote:

could you try the master branch to verify that? Maybe the master branch should be tagged more clear - sorry. 1.6.24 should follow soon

— Reply to this email directly, view it on GitHub https://github.com/phaag/nfdump/issues/333#issuecomment-1079282122, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHUIHYBIUHB3AKJWMN5OF7LVBX7FPANCNFSM5RHQOEPQ . You are receiving this because you authored the thread.Message ID: @.***>

-- Ken Adey

CyGlass, Inc. is a wholly-owned subsidiary of Nominet UK. Nominet UK is registered in England and Wales No. 3203859

This message is intended exclusively for the individual(s) to whom it is addressed and may contain information that is privileged, or confidential. If you are not the addressee, you must not read, use or disclose the contents of this e-mail. If you receive this e-mail in error, please advise us immediately and delete the e-mail. CyGlass, Inc. has taken every reasonable precaution to ensure that any attachment to this e-mail has been swept for viruses. However, Nominet cannot accept liability for any damage sustained as a result of software viruses and would advise that you carry out your own virus checks before opening any attachment

phaag commented 2 years ago

The pcap-reader is just a simple module for debugging purpose and does not handle all possible options. You need either record directly on an ethernet or convert the pcap after collection:

tcprewrite --dlt=enet --enet-dmac=52:54:00:11:11:11 --enet-smac=52:54:00:22:22:22 -i input.pcap -o output.pcap
phaag commented 2 years ago

Fixed in master branch. Palo Alto event does not send ICMP as event FNF type/code but flow ICMP.

ken-adey commented 1 year ago

Peter, I was able to convert the pcap and process it with nfcapd/nfdump.

The issue I'm seeing is for flows from router source 10.11.2.4 . You're right that the type/code is getting through for flows from 10.11.0.4 (as you referenced earlier in the thread).

On Sat, Mar 26, 2022 at 6:56 AM Peter Haag @.***> wrote:

The pcap-reader is just a simple module for debugging purpose and does not handle all possible options. You need either record directly on an ethernet or convert the pcap after collection:

tcprewrite --dlt=enet --enet-dmac=52:54:00:11:11:11 --enet-smac=52:54:00:22:22:22 -i input.pcap -o output.pcap

— Reply to this email directly, view it on GitHub https://github.com/phaag/nfdump/issues/333#issuecomment-1079661635, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHUIHYGTLGIUHVMG3UJVKITVB3UL5ANCNFSM5RHQOEPQ . You are receiving this because you authored the thread.Message ID: @.***>

-- Ken Adey

CyGlass, Inc. is a wholly-owned subsidiary of Nominet UK. Nominet UK is registered in England and Wales No. 3203859

This message is intended exclusively for the individual(s) to whom it is addressed and may contain information that is privileged, or confidential. If you are not the addressee, you must not read, use or disclose the contents of this e-mail. If you receive this e-mail in error, please advise us immediately and delete the e-mail. CyGlass, Inc. has taken every reasonable precaution to ensure that any attachment to this e-mail has been swept for viruses. However, Nominet cannot accept liability for any damage sustained as a result of software viruses and would advise that you carry out your own virus checks before opening any attachment