search

stratosphereips / StratosphereLinuxIPS

Slips, a free software behavioral Python intrusion prevention system (IDS/IPS) that uses machine learning to detect malicious behaviors in the network traffic. Stratosphere Laboratory, AIC, FEL, CVUT in Prague.
Other
688 stars 167 forks source link

Error in flow module trying to convert to int the string of the state of flows. #314

Closed eldraco closed 1 year ago

eldraco commented 1 year ago

Describe the bug When processing certain Zeek files from a folder, there is an error in Error in process_features() of 'Flow ML Detection' module with description "ValueError: could not convert string to float: 'Not Established'"

I'm not sure why the state of a flow is attempted to be converted into an int, but it should not.

To Reproduce Steps to reproduce the behavior:

  1. Go to branch master, version 1.0.3, commit 99d9c3b7
  2. run the command shown below
  3. See error

Expected behavior Not an error processing the state

Screenshots ./slips.py -f /data/My-notebook-normal-2023-02-20-1/ -w Slips. Version 1.0.3 (99d9c3b7) https://stratosphereips.org

[Main] Using redis server on port: 6379 Started Main process [PID 48] Started Output Process [PID 61] Starting modules Something wrong happened while importing the module modules.rnn-cc-detection.rnn-cc-detection: No module named 'tensorflow' Starting the module Risk IQ (Module to get passive DNS info about IPs from RiskIQ) [PID 72] Starting the module ARP (Detect arp attacks) [PID 73] Starting the module Flow Alerts (Alerts about flows: long connection, successful ssh, password guessing, self-signed certificate, data exfiltration, etc.) [PID 75] Starting the module Flow ML Detection (Train or test a Machine Learning model to detect malicious flows) [PID 77] Starting the module HTTP Analyzer (Analyze HTTP flows) [PID 79] Starting the module IP Info (Get different info about an IP/MAC address) [PID 80] Starting the module Network Discovery (Detect Horizonal, Vertical Port scans, ICMP, and DHCP scans) [PID 81] Starting the module Threat Intelligence (Check if the source IP or destination IP are in a malicious list of IPs) [PID 84] Starting the module Timeline (Creates kalipso timeline of what happened in the network based on flows and available data) [PID 86] Starting the module Update Manager (Update Threat Intelligence files) [PID 88] Starting the module Virustotal (IP, domain and file hash lookup on Virustotal) [PID 90]

[Main] Disabled Modules: ['template', 'ensembling', 'exporting_alerts', 'p2ptrust', 'CESNET', 'blocking', 'leak_detector'] [EvidenceProcess-13] Storing Slips logs in output/My-notebook-normal-2023-02-20-1_2023-04-19_08:05:57/ Started Evidence Process [PID 94] Started Profiler Process [PID 95] [Main] Metadata added to output/My-notebook-normal-2023-02-20-1_2023-04-19_08:05:57/metadata Started Input Process [PID 98] [Flow ML Detection] Error in process_features() Flows processed: 0%| | 2/69102, Total analyzed IPs: 0. IPs sending traffic in the last 1 hr: 0. (2023/04/19 08:06:06)Traceback (most recent call last): File "/StratosphereLinuxIPS/modules/flowmldetection/flowmldetection.py", line 166, in process_features dataset.state = dataset.state.astype('float64') File "/usr/local/lib/python3.8/dist-packages/pandas/core/generic.py", line 6324, in astype new_data = self._mgr.astype(dtype=dtype, copy=copy, errors=errors) File "/usr/local/lib/python3.8/dist-packages/pandas/core/internals/managers.py", line 451, in astype return self.apply( File "/usr/local/lib/python3.8/dist-packages/pandas/core/internals/managers.py", line 352, in apply applied = getattr(b, f)(**kwargs) File "/usr/local/lib/python3.8/dist-packages/pandas/core/internals/blocks.py", line 511, in astype new_values = astype_array_safe(values, dtype, copy=copy, errors=errors) File "/usr/local/lib/python3.8/dist-packages/pandas/core/dtypes/astype.py", line 242, in astype_array_safe new_values = astype_array(values, dtype, copy=copy) File "/usr/local/lib/python3.8/dist-packages/pandas/core/dtypes/astype.py", line 187, in astype_array values = _astype_nansafe(values, dtype, copy=copy) File "/usr/local/lib/python3.8/dist-packages/pandas/core/dtypes/astype.py", line 138, in _astype_nansafe return arr.astype(dtype, copy=True) ValueError: could not convert string to float: 'Not Established'

Branch Master

Environment (please complete the following information):

prakharguptaujjain commented 1 year ago

Could you please, if possible, provide the files inside /data/My-notebook-normal-2023-02-20-1/

eldraco commented 1 year ago

Hi @prakharguptaujjain , unfortunately those files were private and I couldn't share. This is an issue for us. Sometimes we try in our own traffic and then we can not share. I'm not sure how other projects are doing this. Any idea?

Anyway I saw that the error was in an update of the pandas library so I fix it already