search

krmaxwell / maltrieve

A tool to retrieve malware directly from the source for security researchers.
GNU General Public License v3.0
562 stars 184 forks source link

exception error on BadStatusLine #19

Closed beamzer closed 10 years ago

beamzer commented 11 years ago

running on Ubuntu 13.04 (running on VMware Workstation) it downloaded just short of 500 items and than:

2013-07-15 17:37:55 -1239418048 Fetched URL hXXp://removebugs.com/4YNN66tx.exe from queue 2013-07-15 17:37:56 -1239418048 Fetched URL hXXp://angelibo.com/1372809886_0.10307000.exe from queue Exception in thread Thread-3: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 810, in bootstrap_inner self.run() File "/usr/lib/python2.7/threading.py", line 763, in run self.__target(_self.args, _self.__kwargs) File "maltrieve.py", line 51, in get_malware mal = get_URL(url) File "/home/beamzer/maltrieve-master/malutil.py", line 7, in get_URL response = urllib2.urlopen(url.encode("utf8")) File "/usr/lib/python2.7/urllib2.py", line 127, in urlopen return _opener.open(url, data, timeout) File "/usr/lib/python2.7/urllib2.py", line 404, in open response = self._open(req, data) File "/usr/lib/python2.7/urllib2.py", line 422, in _open '_open', req) File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain result = func(args) File "/usr/lib/python2.7/urllib2.py", line 1214, in http_open return self.do_open(httplib.HTTPConnection, req) File "/usr/lib/python2.7/urllib2.py", line 1187, in do_open r = h.getresponse(buffering=True) File "/usr/lib/python2.7/httplib.py", line 1045, in getresponse response.begin() File "/usr/lib/python2.7/httplib.py", line 409, in begin version, status, reason = self._read_status() File "/usr/lib/python2.7/httplib.py", line 373, in _read_status raise BadStatusLine(line) BadStatusLine: ''

followed by a couple more downloads and another:

2013-07-15 17:37:57 -1239418048 Fetched URL hXXp://host0r.net/xs.exe from queue 2013-07-15 17:37:58 -1239418048 Fetched URL hXXp://www.sineglu.it/jUnejSe.exe from queue 2013-07-15 17:37:58 -1239418048 Fetched URL hXXp://sourcehonduras.net/load/magic_school_bus_videos_online_free.exe from queue Exception in thread Thread-2: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 810, in bootstrap_inner self.run() File "/usr/lib/python2.7/threading.py", line 763, in run self.__target(_self.args, _self.__kwargs) File "maltrieve.py", line 51, in get_malware mal = get_URL(url) File "/home/beamzer/maltrieve-master/malutil.py", line 7, in get_URL response = urllib2.urlopen(url.encode("utf8")) File "/usr/lib/python2.7/urllib2.py", line 127, in urlopen return _opener.open(url, data, timeout) File "/usr/lib/python2.7/urllib2.py", line 404, in open response = self._open(req, data) File "/usr/lib/python2.7/urllib2.py", line 422, in _open '_open', req) File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain result = func(args) File "/usr/lib/python2.7/urllib2.py", line 1214, in http_open return self.do_open(httplib.HTTPConnection, req) File "/usr/lib/python2.7/urllib2.py", line 1187, in do_open r = h.getresponse(buffering=True) File "/usr/lib/python2.7/httplib.py", line 1045, in getresponse response.begin() File "/usr/lib/python2.7/httplib.py", line 409, in begin version, status, reason = self._read_status() File "/usr/lib/python2.7/httplib.py", line 373, in _read_status raise BadStatusLine(line) BadStatusLine: ''

and after that it seems to hang :-( lsof shows:

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME python 10353 beamzer cwd DIR 8,1 4096 525583 /home/beamzer/maltrieve-master python 10353 beamzer rtd DIR 8,1 4096 2 / python 10353 beamzer txt REG 8,1 2688640 1048748 /usr/bin/python2.7 python 10353 beamzer mem REG 8,1 161156 917896 /lib/i386-linux-gnu/libexpat.so.1.6.0 python 10353 beamzer mem REG 8,1 83816 917882 /lib/i386-linux-gnu/libresolv-2.17.so python 10353 beamzer mem REG 8,1 22100 917886 /lib/i386-linux-gnu/libnss_dns-2.17.so python 10353 beamzer mem REG 8,1 9660 917838 /lib/libnss_mdns4_minimal.so.2 python 10353 beamzer mem REG 8,1 47080 917774 /lib/i386-linux-gnu/libnss_files-2.17.so python 10353 beamzer mem REG 8,1 153048 917593 /lib/i386-linux-gnu/liblzma.so.5.0.0 python 10353 beamzer mem REG 8,1 537016 917908 /lib/i386-linux-gnu/libgcrypt.so.11.7.0 python 10353 beamzer mem REG 8,1 1396864 1053656 /usr/lib/i386-linux-gnu/libxml2.so.2.9.0 python 10353 beamzer mem REG 8,1 83204 1055168 /usr/lib/i386-linux-gnu/libexslt.so.0.8.16 python 10353 beamzer mem REG 8,1 243152 1055174 /usr/lib/i386-linux-gnu/libxslt.so.1.1.27 python 10353 beamzer mem REG 8,1 9664 917840 /lib/libnss_mdns4.so.2 python 10353 beamzer mem REG 8,1 58344 132814 /usr/lib/python2.7/lib-dynload/pyexpat.i386-linux-gnu.so python 10353 beamzer mem REG 8,1 1255612 139605 /usr/lib/python2.7/dist-packages/lxml/etree.so python 10353 beamzer mem REG 8,1 88476 131818 /usr/lib/python2.7/lib-dynload/datetime.i386-linux-gnu.so python 10353 beamzer mem REG 8,1 350296 917993 /lib/i386-linux-gnu/libssl.so.1.0.0 python 10353 beamzer mem REG 8,1 1734784 917994 /lib/i386-linux-gnu/libcrypto.so.1.0.0 python 10353 beamzer mem REG 8,1 26256 1182510 /usr/lib/i386-linux-gnu/gconv/gconv-modules.cache python 10353 beamzer mem REG 8,1 48328 132825 /usr/lib/python2.7/lib-dynload/_json.i386-linux-gnu.so python 10353 beamzer mem REG 8,1 2932160 1180296 /usr/lib/locale/locale-archive python 10353 beamzer mem REG 8,1 267816 917771 /lib/i386-linux-gnu/libm-2.17.so python 10353 beamzer mem REG 8,1 1770984 917776 /lib/i386-linux-gnu/libc-2.17.so python 10353 beamzer mem REG 8,1 95764 917900 /lib/i386-linux-gnu/libz.so.1.2.7 python 10353 beamzer mem REG 8,1 9816 917883 /lib/i386-linux-gnu/libutil-2.17.so python 10353 beamzer mem REG 8,1 13856 917880 /lib/i386-linux-gnu/libdl-2.17.so python 10353 beamzer mem REG 8,1 124637 917784 /lib/i386-linux-gnu/libpthread-2.17.so python 10353 beamzer mem REG 8,1 13644 918512 /lib/i386-linux-gnu/libgpg-error.so.0.8.0 python 10353 beamzer mem REG 8,1 32788 132671 /usr/lib/python2.7/lib-dynload/_ssl.i386-linux-gnu.so python 10353 beamzer mem REG 8,1 15336 132662 /usr/lib/python2.7/lib-dynload/_hashlib.i386-linux-gnu.so python 10353 beamzer mem REG 8,1 134376 917884 /lib/i386-linux-gnu/ld-2.17.so python 10353 beamzer 0u CHR 136,1 0t0 4 /dev/pts/1 python 10353 beamzer 1u CHR 136,1 0t0 4 /dev/pts/1 python 10353 beamzer 2u CHR 136,1 0t0 4 /dev/pts/1 python 10353 beamzer 3u IPv4 1273846 0t0 TCP 192.168.253.128:33955->61.187.182.21:http (ESTABLISHED) python 10353 beamzer 4u IPv4 1272841 0t0 TCP 192.168.253.128:46383->222.186.33.73:http (ESTABLISHED)

krmaxwell commented 11 years ago

Issue #22 should help with resolving this (because we need to log more info to see what's happening).

Regardless, needs better exception handling!