Closed ameya-joshi-tuc closed 5 years ago
Hi @AmJo29 I am having exactly the same issue. I am trying to generate UDP and also want to generate features for packets which are not complete. But I am getting zero packets generated for those! Currently, I see it working only for packets which are complete!
@kshntn @AmJo29 Please upload your sample PCAP in order to investigate more (I do not have any problem with UDP). @kshntn The logic for closing a Netflow is defined in addPacket function in FlowGenerator.java. In order to capture flows that do not have FIN flag (packet.hasFlagFIN()), you should reduce the Netflow timeout (currently, it is 2 minutes). You can find it on line 67 of CICFlowMeter.java file: flowGen = new FlowGenerator(true,120000000L, 5000000L);
Hi @imanoracle I generated a pcap file for a DDOS attack.. It has packets which is not complete. I have uploaded it here. Also I will send the .pcap for udp in some time! notworking.pcap.zip Can you tell me if its working. Or is it the issue of the pcap file!
@imanoracle Thank you for the reply. I cannot upload the files unfortunately. I did experiment with the UDP Timeouts in offline mode and it did not work, but thanks for pointing out the line number in the file, I will see if that helps me with the online phase. Maybe if it turns out that the problem is in the PCAP files (@kshntn ), I will generate a new set of packets on a different system and see if that helps
Hi @AmJo29 I had some UDP packets in a pcap file. but the features were only generated for the bidirectional TCP packets!
Yes that is the issue for me as well. Since UDP doesn't include the usual TCP handshakes for establishing a connection and FIN (As indicated by @imanoracle) to terminate the connection, we can only give it timeouts and ask it to calculate the flow statistics for 'that' timeout window between a source and a destination. Hasn't worked for me though sadly
Hi @AmJo29 thats right. I tried the way @imanoracle mentioned, but it was not successful.. I guess my pcap file has some errors or I did not do it right! Could you please tell me whether the pcap file i uploaded is okay ? Can I have some way to contact you faster (email id or so?).
I think GitHub is probably the fastest way tbh. Also, there are no UDP packets in your PCAP file
No @AmJo29 .. Not this one.. This one is for DDOS attacks using TCP SYN flood. This one has packets which are not complete. I wanted features for this too!
working.pcap.zip This one is a pcap with TCP and UDP packets and this generates 30 flows which are only for bidirectional TCP packets. Could you check it at your end too!
@kshntn I was able to generate the UDP flows for your PCAP file (working.pcap) I took a random UDP timestamp (at around time 20) The next non-UDP flow occurs at timestamp 25 so in CICFlowMeter, I set the Timeout values to 6000000 (Both timeout fields; the timeout unit is in microseconds) and was able to generate 82 Flows which includes both UDP and TCP. I confirmed it was the same flow by checking the source and destination addresses and ports but did not have the time to verify if the values generated by CICFlowMeter are correct (you can try that out). Try the steps I mentioned at your end and see if it works for you, if not maybe you will have to reinstall the application or something
Hi @AmJo29 I tried changing the timeout values to 6000000 and then tried creating CSV files. The number of flows was changing each time I executed it. Is that expected?
If it is maintained at 6000000, according to me it should not. If you are changing the values, then yes it can change. I suggest you go over the CICFlowMeter and related papers, you will understand it better - http://www.netflowmeter.ca/netflowmeter.html
Thanks @AmJo29 I will go through it and let you know.. Did you change the timeout values in the code or the directly in the tool?
Since I checked your PCAP files which is offline mode, I changed it with the tool. If you need it real-time, you will have to modify the code Try and see if the changes work for you. It hasn't worked for me still but if it works for you, the issue might be at my end, so do post if it worked and what changes you did so that the issue can be marked with your solution as closed by the mods (I am new to GitHub, I do not know how this works)
I changed it in the code. I will try again and let you know. But I am also looking for features for pcap with incomplete packets. Can you also tell me if the notworking pcap which I uploaded some time back can also be used to generate features?
See the links, you will understand. The best way is to simply experiment and see what all works. Since I am a fellow user like you and not the developer, there is only so much that I know and can help you with :)
Thank you @AmJo29 . I am going through the papers.. :+1:
Mention not. All the best and do post a solution if the problem is solved so that the mods can close the thread :)
Hi!
I have one question related to extracting .CSV dataset from PCAP file, in order to be able to continue my work. I will greatly appreciate your answer.
I need to extract time series dataset from pcap file with a time window of 10 minutes.
would you help me how change the time windows in CICFlowMeter source code? and whats the default value?
Also, are you have any idea if there any option in TSHARK or tranalyzer to do the same?
Mahmoud
Hi, We are pleased to announce that the first version of the Network and Transportation Layers Flow Analyzer (NTLFlowLyzer) is now available as part of the Understanding Cybersecurity Series (UCS) knowledge mobilization program. This Python open-source project has been designed and developed to address many issues encountered with CICFlowMeter. It extracts over 300 features from TCP-based network traffic, tailored explicitly for Anomaly Profiling (AP). NTLFlowLyzer serves as a key component of the upcoming NetFlowLyzer.
https://github.com/ahlashkari/NTLFlowLyzer
Best, BCCC Team https://www.yorku.ca/research/bccc/
Hello
I am unable to obtain UDP Flows using CICFlowMeter in both Online/Offline mode. I made a pcap capture with Wireshark to try and extract UDP flows using CICFlowMeter but it only creates the TCP flows and not the UDP ones. Also, is there a way to obtain ICMP flows between two IP addresses. I mostly use Python, not Java so I don't know how or where I can make changes for controlling UDP timeouts and capturing ICMP flows but if someone can point me in the right direction, I will be extremely grateful. More important is the UDP issue. (I unfortunately cannot upload the exact files)
All captures are carried out on a Virtual Machine running Ubuntu 16.04 LTS
I thank you for your help