search

odedshimon / BruteShark

Network Analysis Tool
GNU General Public License v3.0
2.98k stars 324 forks source link

BruteSharkCli is killed when trying to use network map module on a directory of pcaps, no output created #122

Open syloktools opened 2 years ago

syloktools commented 2 years ago

Output from terminal:

xxx@xxx:/xxx$ sudo ./BruteSharkCli -m NetworkMap -d /xxx/data/packets/servers/dailylogs/2019-05-02/ -o /xxx/results [+] Start analyzing 11 files [+] Start processing file : daemonlogger.pcap.1556805601 [+] Finished processing file : daemonlogger.pcap.1556805601 [+] Start processing file : daemonlogger.pcap.1556816401 [+] Finished processing file : daemonlogger.pcap.1556816401 [+] Start processing file : daemonlogger.pcap.1556820001 [+] Finished processing file : daemonlogger.pcap.1556820001 [+] Start processing file : daemonlogger.pcap.1556803424 [+] Finished processing file : daemonlogger.pcap.1556803424 [+] Start processing file : daemonlogger.pcap.1556809201 [+] Finished processing file : daemonlogger.pcap.1556809201 [+] Start processing file : daemonlogger.pcap.1556830801 [+] Finished processing file : daemonlogger.pcap.1556830801 [+] Start processing file : daemonlogger.pcap.1556827201 [+] Finished processing file : daemonlogger.pcap.1556827201 [+] Start processing file : daemonlogger.pcap.1556834401 Killed

If I run it against one file: xxx@xxx:/xxx$ ./BruteSharkCli -m NetworkMap -i /xxx/data/packets/servers/dailylogs/2019-05-02/daemonlogger.pcap.1556812801 -o /xxx/ [+] Start analyzing 1 files [+] Start processing file : daemonlogger.pcap.1556812801 [+] Finished processing file : daemonlogger.pcap.1556812801 [+] Successfully exported network map to json file: /xxx/resultsBruteShark Network Map.json [+] Successfully exported network nodes data to json file: /xxx/BruteShark Network Nodes Data.json [+] Successfully exported extracted files to: /xxx/Files [+] BruteShark finished processing

Size of directory is 912M. Is there a limit. System has 8 gig of memory.

odedshimon commented 2 years ago

Hi @robertnixon2003 ! Thanks for creating this issue. There is no built-in limitation at BruteShark, also there is no log containing the phrase "killed". Therefore I tend to believe it some kind of operating system lack of resource.

I can suggest few ways to investigate the issue:

  1. Make sure the folder exists and it has write permissions.
  2. Run all files one by one to ensure that this behavior is not related to a specific file.
  3. Clone this project and run it at debug mode (for accurate exception and stack trace).

Feel free to contact with any further questions

fariaalex commented 2 years ago

Hi guys, I also encountered the same problem. See below 

jan 26 19:56:05 qa-br-vostro kernel: Tasks state (memory values in pages):
jan 26 19:56:05 qa-br-vostro kernel: [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
jan 26 19:56:05 qa-br-vostro kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=user.slice,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/user@1000.service/app.slice/vte-spawn->
jan 26 19:56:05 qa-br-vostro kernel: Out of memory: Killed process 16242 (BruteSharkCli) total-vm:10105664kB, anon-rss:4961504kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:10280kB oom_score_adj:0
jan 26 19:56:05 qa-br-vostro kernel: oom_reaper: reaped process 16242 (BruteSharkCli), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
jan 26 19:56:05 qa-br-vostro systemd[1]: user@1000.service: A process of this unit has been killed by the OOM killer.
jan 26 19:56:05 qa-br-vostro systemd[1504]: vte-spawn-48f92492-8cbe-4dec-90b7-ef408d10d774.scope: A process of this unit has been killed by the OOM killer.

File:
-rw-r--r-- 1 g0043780 g0043780 385M jan 26 17:01 Boot-2601-all.pcapng

cmd:
./BruteSharkCli -i ../Plataformas/Boot-2601-all.pcapng -o ../Plataformas/

 ./BruteSharkCli --version
BruteSharkCli 1.0.0.0

free -h
               total        used        free      shared  buff/cache   available
Mem.:          7,6Gi       4,4Gi       217Mi       706Mi       3,1Gi       2,3Gi
Swap:          975Mi       480Mi       495Mi

uname -a
Linux qa-br-vostro 5.10.0-11-amd64 #1 SMP Debian 5.10.92-1 (2022-01-18) x86_64 GNU/Linux

Strace (last lines):

sysinfo({uptime=17516, loads=[93664, 89088, 79360], totalram=8205803520, freeram=3322908672, sharedram=230916096, bufferram=6385664, totalswap=1023406080, freeswap=4096, procs=971, totalhigh=0, freehigh=0, mem_unit=1}) = 0
sysinfo({uptime=17516, loads=[93664, 89088, 79360], totalram=8205803520, freeram=3322908672, sharedram=230916096, bufferram=6385664, totalswap=1023406080, freeswap=4096, procs=971, totalhigh=0, freehigh=0, mem_unit=1}) = 0
openat(AT_FDCWD, "/proc/meminfo", O_RDONLY) = 59
fstat(59, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
read(59, "MemTotal:        8013480 kB\nMemF"..., 1024) = 1024
close(59)                               = 0
mprotect(0x7fbe26866000, 4284416, PROT_READ|PROT_WRITE) = 0
read(53, "\266o\321\302m\t\25X\344\202\6\35\265\204XB\376$a\333\202J\n\230\20\376\301\253[\370C5"..., 4096) = 4096
sysinfo({uptime=17516, loads=[93664, 89088, 79360], totalram=8205803520, freeram=3318521856, sharedram=230916096, bufferram=6385664, totalswap=1023406080, freeswap=4096, procs=971, totalhigh=0, freehigh=0, mem_unit=1}) = 0
sysinfo({uptime=17516, loads=[93664, 89088, 79360], totalram=8205803520, freeram=3318521856, sharedram=230916096, bufferram=6385664, totalswap=1023406080, freeswap=4096, procs=971, totalhigh=0, freehigh=0, mem_unit=1}) = 0
openat(AT_FDCWD, "/proc/meminfo", O_RDONLY) = 59
fstat(59, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
read(59, "MemTotal:        8013480 kB\nMemF"..., 1024) = 1024
close(59)                               = 0
mprotect(0x7fbcc2650000, 91545600, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbd6ffec000
mprotect(0x7fbd6ffec000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd4e8da000, 2093056, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd6ffee000, 140009472, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd8577d000, 139882496, PROT_READ|PROT_WRITE) = 0
read(53, "\272\275k/\vd\356,\210Ww\364\233\24Sj\351\207\23\257#\354\232\332\230\2\273\225L\373\256\315"..., 4096) = 4096
mprotect(0x7fbd227b1000, 91676672, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbd5ffea000
mprotect(0x7fbd5ffea000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd5ffec000, 140013568, PROT_READ|PROT_WRITE) = 0
read(53, "\214\5\0\0\0\0\0\0s\352\315\26\264\351\t@j\5\0\0j\5\0\0\1\0^\0\2=\30J"..., 4096) = 4096
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbd4ffe8000
mprotect(0x7fbd4ffe8000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd4ffea000, 140013568, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbdd0582000, 91676672, PROT_READ|PROT_WRITE) = 0
read(53, "ramData> </CustomParams> </Sched"..., 4096) = 4096
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbd3a22e000
mprotect(0x7fbd3a22e000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd3a230000, 140017664, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbda857a000, 91676672, PROT_READ|PROT_WRITE) = 0
read(53, " </CustomParams> </ScheduleEvent"..., 4096) = 4096
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbd2a22c000
mprotect(0x7fbd2a22c000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd2a22e000, 140017664, PROT_READ|PROT_WRITE) = 0
read(53, "x\361\22\211\275\206\265\255\30\343\22\21\303\35(\326\335\255s\272\253\252\241\260\322\226\24$\322h&\316"..., 4096) = 4096
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbcfa21e000
mprotect(0x7fbcfa21e000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbcfa220000, 140017664, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd98578000, 91680768, PROT_READ|PROT_WRITE) = 0
read(53, "vent> <ScheduleEvent> <InstanceD"..., 4096) = 4096
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbcda21a000
mprotect(0x7fbcda21a000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbcda21c000, 140021760, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd78574000, 91684864, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbcca218000
mprotect(0x7fbcca218000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbcca21a000, 140021760, PROT_READ|PROT_WRITE) = 0
read(53, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096) = 4096
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbcaa214000
mprotect(0x7fbcaa214000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbcaa216000, 140021760, PROT_READ|PROT_WRITE) = 0
read(53, "\350IoR\265)G\214\204\325\0364\245\347D\267\370r\220\353\251u\312\256t\235Vi\20\370\20\v"..., 4096) = 4096
mprotect(0x7fbd68573000, 91680768, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc9a212000
mprotect(0x7fbc9a212000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbc9a214000, 140021760, PROT_READ|PROT_WRITE) = 0
read(53, "\340\352\305;\270\352K\202\337\365\373\244\323\215\367n\370\267\355N\240P0\242\305\233:S\321\240\365\254"..., 4096) = 4096
mprotect(0x7fbd58571000, 91684864, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc8a210000
mprotect(0x7fbc8a210000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbc8a212000, 140025856, PROT_READ|PROT_WRITE) = 0
read(53, "\365\357\240\271\245dr\24#u\311U8\356\231EH\301\203\1\330\344_+\327QU`a\r\364?"..., 4096) = 4096
mprotect(0x7fbdb860a000, 65536, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc7a20e000
mprotect(0x7fbc7a20e000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbc7a210000, 140025856, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd427b8000, 91684864, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc6a20c000
mprotect(0x7fbc6a20c000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbc6a20e000, 140025856, PROT_READ|PROT_WRITE) = 0
read(53, "\27\7\0\4\0\0GC\351\30\0\0\1\340\30)\204\320\r=\256}\23?\35\256}\7\205\200\262\267"..., 4096) = 4096
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc5a20a000
mprotect(0x7fbc5a20a000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbc5a20c000, 140029952, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd327b6000, 91684864, PROT_READ|PROT_WRITE) = 0
read(53, "\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377\377"..., 4096) = 4096
mprotect(0x7fbdb861a000, 65536, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc4a208000
mprotect(0x7fbc4a208000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbc4a20a000, 140029952, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbd027a8000, 91688960, PROT_READ|PROT_WRITE) = 0
read(53, "\330\326\205]\220jY\327o\212X\251{\235h\272\250\245w\315\232\374\350\334\213;\f\200\16\352\226\3"..., 4096) = 4096
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc3a206000
mprotect(0x7fbc3a206000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbc3a208000, 140029952, PROT_READ|PROT_WRITE) = 0
mmap(NULL, 268443648, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbc2a204000
mprotect(0x7fbc2a204000, 8192, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbc2a206000, 140034048, PROT_READ|PROT_WRITE) = 0
read(53, "o\211\250\304\247\352\16u\224\340v\10#)1{\f\335=\243\230U$\242\213\207\201\2h\256T\216"..., 4096) = 4096
mprotect(0x7fbdb862a000, 65536, PROT_READ|PROT_WRITE) = 0
mprotect(0x7fbce27a5000, 91688960, PROT_READ|PROT_WRITE) = 0
+++ killed by SIGKILL +++
Morto