search

robcowart / elastiflow

Network flow analytics (Netflow, sFlow and IPFIX) with the Elastic Stack
Other
2.48k stars 590 forks source link

NetStream is supports #638

Closed luweijun1992 closed 2 years ago

luweijun1992 commented 3 years ago

Huawei USG firewall only supports NetStrean. Can ElastiFlow support NetStream traffic analysis? Is it possible to use port 2055 for flow analysis of NetStream? I do this, but the traffic I get doesn't match what I actually get. image

image

robcowart commented 3 years ago

My expectation is that it should work. If you can send me a PCAP of the flow records I can investigate it further. You can send it to elastiflow@gmail.com rather than post it here.

luweijun1992 commented 3 years ago

My expectation is that it should work. If you can send me a PCAP of the flow records I can investigate it further. You can send it to elastiflow@gmail.com rather than post it here.

The email has been sent, please check it.

luweijun1992 commented 3 years ago

My expectation is that it should work. If you can send me a PCAP of the flow records I can investigate it further. You can send it to elastiflow@gmail.com rather than post it here.

Do you support it?

robcowart commented 3 years ago

I don't see any issue with the flow records. What you are seeing could be related to the difference between the way that Netflow vs. SNMP (I assume that is how the second chart is collected) provide data. You could also check if you are dropping packets. Run netstat -su a couple times and check if you are seeing increasing number of Receive Buffer Errors. That can indicate that UDP packets are being dropped before ElastiFlow can process them.

luweijun1992 commented 3 years ago

I don't see any issue with the flow records. What you are seeing could be related to the difference between the way that Netflow vs. SNMP (I assume that is how the second chart is collected) provide data. You could also check if you are dropping packets. Run netstat -su a couple times and check if you are seeing increasing number of Receive Buffer Errors. That can indicate that UDP packets are being dropped before ElastiFlow can process them.

netstat -su The count is always 0 image If you look at the information from Top minus N, the bandwidth shouldn't be that low image

samtaoys commented 3 years ago

I don't see any issue with the flow records. What you are seeing could be related to the difference between the way that Netflow vs. SNMP (I assume that is how the second chart is collected) provide data. You could also check if you are dropping packets. Run netstat -su a couple times and check if you are seeing increasing number of Receive Buffer Errors. That can indicate that UDP packets are being dropped before ElastiFlow can process them.

I also have the same problem. cause Huawei USG Firewall only support the netstream protocol , I try to use wireshark to get the different，in wireshark the netstream is considered as the netflow protocal ,but the flow structure is different with cisco netflow so when I set the 2055 UDP port to the elastiflow server , It can receive the discovery it but it can't analyzer them. Will this project add to support netstream protocol in the furture ? netstream netflow

luweijun1992 commented 3 years ago

I don't see any issue with the flow records. What you are seeing could be related to the difference between the way that Netflow vs. SNMP (I assume that is how the second chart is collected) provide data. You could also check if you are dropping packets. Run netstat -su a couple times and check if you are seeing increasing number of Receive Buffer Errors. That can indicate that UDP packets are being dropped before ElastiFlow can process them.

I also have the same problem. cause Huawei USG Firewall only support the netstream protocol , I try to use wireshark to get the different，in wireshark the netstream is considered as the netflow protocal ,but the flow structure is different with cisco netflow so when I set the 2055 UDP port to the elastiflow server , It can receive the discovery it but it can't analyzer them. Will this project add to support netstream protocol in the furture ? netstream netflow

I think you should be from China, the following reply will be in Chinese. 华为发出的NetStream报文没有发送采样比导致elastiflow分析出来的数据不准，针对v4版本可用手动配置采样比（logstash/elastiflow/user_settings/app_id.srctype.yml），现在v5-beta版本暂时还不支持（作者回复未来会支持）。

lukedevon commented 3 years ago

Hi, I also trying to integrate Huawei - Netstream protocol by using elastiflow project. May I check with you, does elastiflow support for Netstream as well?

Thanks Luke.

robcowart commented 3 years ago

It should work fine. The only issue with Huawei is that they often don't include the sample rate in the record, so this must be defined manually per device in the ElastiFlow configuration.

robcowart commented 2 years ago

This issue is being closed as this legacy version of ElastiFlow is now deprecated and is to be archived. Please try the new ElastiFlow, request a free Basic Tier license, and join the ElastiFlow Community Slack. Thank you.