moonshawdo / checkgoogleip

主要是用来检查哪些IP可以用在goagent上面
Apache License 2.0
581 stars 238 forks source link

扫描多个文件时读取不到IP段 #43

Closed xyuanmu closed 9 years ago

xyuanmu commented 9 years ago

因为扫描IP库过大时会内存不够,于是就分割IP段为googleip-1.txt、googleip-2.txt、googleip-3.txt,可是扫描的时候,第一个文件扫描完,第二个开始就:ip queue size: 0,没法扫描,请问下是否有办法可以解决?

begin check googleip-2.txt
[Thread-52]begin to get ramdom ip
[MainThread]need create max threads count: 50
[Thread-52]load extra ip ok,line:1,load default ip: 0
[Thread-52]ramdom ip thread stopped.had check ip: 0,rest ip queue size: 0
[MainThread]try to collect ssl result,check ip cnt: 256,times:0s
[MainThread]write to file D:\Portable\GoAgent\checkiptools\ip.txt ok,count:0

begin check googleip-3.txt
[Thread-103]begin to get ramdom ip
[MainThread]need create max threads count: 50
[Thread-103]load extra ip ok,line:5,load default ip: 0
[Thread-103]ramdom ip thread stopped.had check ip: 0,rest ip queue size: 0
[MainThread]try to collect ssl result,check ip cnt: 256,times:0s
[MainThread]write to file D:\Portable\GoAgent\checkiptools\ip.txt ok,count:0

更改的代码从1060行开始:

if __name__ == '__main__':
    if len(sys.argv) > 1:
        checkip(sys.argv[1])
    else:
        files = os.listdir(g_filedir)
        i = 0
        for item in files:
            if "googleip-" in item:
                i = re.findall(r'([0-9]+)',item)[0]
                if os.path.exists("googleip-%s.txt" % i):
                    g_googleipfile = os.path.join(g_filedir,"googleip-%s.txt" % i)
                    print "\nbegin check googleip-%s.txt" % i
                    list_ping()
                    if not os.path.exists(g_filedir + "/tmp"): os.mkdir("tmp")
                    if os.path.exists("ip_tmperror.txt"): os.rename("ip_tmperror.txt", "ip_tmperror-%s.txt" % i)
                    if os.path.exists("ip_tmpno.txt"): os.rename("ip_tmpno.txt", "ip_tmpno-%s.txt" % i)
                    if os.path.exists("ip_tmpok.txt"): os.rename("ip_tmpok.txt", "ip_tmpok-%s.txt" % i)
                    if os.path.exists("ip_tmperror-%s.txt" % i): shutil.move("ip_tmperror-%s.txt" % i, "tmp/")
                    if os.path.exists("ip_tmpno-%s.txt" % i): shutil.move("ip_tmpno-%s.txt" % i, "tmp/")
                    if os.path.exists("ip_tmpok-%s.txt" % i): shutil.move("ip_tmpok-%s.txt" % i, "tmp/")
            elif item == 'googleip.txt' and i == 0:
                list_ping()
xyuanmu commented 9 years ago

知道问题所在了,evt_ipramdomend.clear() 就行啦。不过奇怪的是线程会一直累计,内存是控制住了。