search

facebookarchive / flashback

Capture and replay real mongodb workloads
Other
217 stars 72 forks source link

pcap having parse issues #33

Closed dbmurphy closed 8 years ago

dbmurphy commented 8 years ago

See bolded items below , this was a simple tcpdump -i eth0 'dst port 27017' and it appears only GLE's are parseable. @tredman have any thoughts?

12028 packets captured 12066 packets received by filter 38 packets dropped by kernel [host/user hidden]$ ./gocode/bin/pcap_converter -f mongod2.pcap 2015/11/24 11:29:46 starting stream 35337->27017 35337->27017 2015/11/24 11:29:46 starting stream 36688->27017 36688->27017 2015/11/24 11:29:46 starting stream 41306->27017 41306->27017 2015/11/24 11:29:46 starting stream 47213->27017 47213->27017 2015/11/24 11:29:46 starting stream 53872->27017 53872->27017 2015/11/24 11:29:46 starting stream 37393->27017 37393->27017 2015/11/24 11:29:46 starting stream 40384->27017 40384->27017 2015/11/24 11:29:46 starting stream 55352->27017 55352->27017 2015/11/24 11:29:46 starting stream 43669->27017 43669->27017 2015/11/24 11:29:46 starting stream 56853->27017 56853->27017 2015/11/24 11:29:46 starting stream 34560->27017 34560->27017 2015/11/24 11:29:46 starting stream 51858->27017 51858->27017 2015/11/24 11:29:46 starting stream 41928->27017 41928->27017 2015/11/24 11:29:46 starting stream 36132->27017 36132->27017 2015/11/24 11:29:46 starting stream 56186->27017 56186->27017 2015/11/24 11:29:46 starting stream 47873->27017 47873->27017 2015/11/24 11:29:46 starting stream 49007->27017 49007->27017 2015/11/24 11:29:46 starting stream 58107->27017 58107->27017 2015/11/24 11:29:46 error parsing op: unexpected EOF 2015/11/24 11:29:46 error parsing op: mongoproto: got invalid document size discarded 719272 {"command":{"getlasterror":1,"j":false,"fsync":false,"wtimeout":null},"ns":"apache.$cmd","ntoreturn":-1,"ntoskip":0,"op":"command","ts":{"$date":"1970-01-17T18:19:45.634Z"}} {"command":{"getlasterror":1,"j":false,"fsync":false,"wtimeout":null},"ns":"apache.$cmd","ntoreturn":-1,"ntoskip":0,"op":"command","ts":{"$date":"1970-01-17T18:19:45.634Z"}} {"command":{"getlasterror":1,"j":false,"fsync":false,"wtimeout":null},"ns":"apache.$cmd","ntoreturn":-1,"ntoskip":0,"op":"command","ts":{"$date":"1970-01-17T18:19:45.634Z"}} {"command":{"getlasterror":1,"j":false,"fsync":false,"wtimeout":null},"ns":"apache.$cmd","ntoreturn":-1,"ntoskip":0,"op":"command","ts":{"$date":"1970-01-17T18:19:45.634Z"}} {"command":{"getlasterror":1,"j":false,"fsync":false,"wtimeout":null},"ns":"apache.$cmd","ntoreturn":-1,"ntoskip":0,"op":"command","ts":{"$date":"1970-01-17T18:19:45.634Z"}} {"command":{"getlasterror":1,"j":false,"fsync":false,"wtimeout":null},"ns":"apache.$cmd","ntoreturn":-1,"ntoskip":0,"op":"command","ts":{"$date":"1970-01-17T18:19:45.634Z"}}

tredman commented 8 years ago

Haven't used this in month but I remember it working ok for capturing geo queries. Did you try running a capture with just "port 27017"? I vaguely recall something about how the pcap tools open a go routine for each stream that runs until it detects the end of the stream - maybe just using "dst port 27017" only captures part of the information you need for other ops?

tmc commented 8 years ago

@dbmurphy it might be more fitting to open this on https://github.com/tmc/mongocaputils -- that said, including the packet(s) that are failing would be useful.