search

SuperCowPowers / zat

Zeek Analysis Tools (ZAT): Processing and analysis of Zeek network data with Pandas, scikit-learn, Kafka and Spark
MIT License
423 stars 107 forks source link

http.log file with the size of 2.0 MB zat stuck #138

Closed lokipython closed 2 years ago

lokipython commented 2 years ago

I have a log file http.log file with the size of 2.0MB , my script just hang when i load it to data frames if i give small size file it loads up11 above image its not loading

up44 this one is loading because of size may be , any help so that i can load large files

brifordwylie commented 2 years ago

@lokipython we have read in Zeek log files that are 2-3 Gigabytes (see https://supercowpowers.github.io/zat/large_dataframes.html) so there should not be any problem with a 2MB file, can you give a bit more information on the issue you're seeing. Also you can mail me the file at briford@supercowpowers.com if you'd like me to diagnose the issue.