search

Shotokhan / kanku-sho

Tool for remote capture, traffic analysis and filtering, querying of indexed data and automatic code generation for replaying requests with a Flask interface
GNU General Public License v3.0
4 stars 0 forks source link

Computational complexity for large capture files #2

Closed Shotokhan closed 4 years ago

Shotokhan commented 4 years ago

The "Analysis Controller" module is slow at computing large capture files: it would be fine to have a module to split large files in little files to have a faster feedback from them on the Flask interface. Otherwise, the "Pyshark functions" module could take care of it.

Shotokhan commented 4 years ago

Solved: now the capture on the remote host is not made anymore according to the round time, but it is made in a rotating buffer of files; the number of files and the size of each file must be specified in the configuration file (it has some default values) according to the volume of traffic on the remote host. Plus, files are compressed and the sleep time for the module which downloads files from the remote host has been made adaptive to the speed of the download. The achievements are that the "Analysis Controller" module will work on much tinier files than before, and the traffic is downloaded in a reasonable time (near real-time), allowing the "Analysis Controller" module to not wait for other files to be downloaded. The last thing to state is the addition of a "preserve_all" option in the configuration file: if it is set to True, all capture files will be preserved in "local_pcap_backup", otherwise there will be a buffer also on the analyzer host, resulting in the save of disk space. To avoid many problems related to timing, I suggest to set the "num_circular_files" option to a value high enough to not risk the lose of some files before download / before end of analysis, and to have enough traffic to inspect later on Wireshark even if the "preserve_all" option is set to False.