likexian / whois

Whois client for domain and ip whois information query in Go(Golang).
Apache License 2.0
382 stars 70 forks source link

批量查询时出现 goroutine 907686 [runnable]: #27

Closed namecost closed 1 year ago

namecost commented 1 year ago

goroutine 908118 [IO wait]: internal/poll.runtime_pollWait(0x7fb8d8d13b48, 0x72) /usr/local/go/src/runtime/netpoll.go:302 +0x89 internal/poll.(pollDesc).wait(0xc020a74880?, 0xc01f233200?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32 internal/poll.(pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:88 internal/poll.(FD).Read(0xc020a74880, {0xc01f233200, 0x200, 0x200}) /usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a net.(netFD).Read(0xc020a74880, {0xc01f233200?, 0xc004c4f458?, 0x44be12?}) /usr/local/go/src/net/fd_posix.go:55 +0x29 net.(conn).Read(0xc011e78848, {0xc01f233200?, 0xe359636833?, 0xda1160?}) /usr/local/go/src/net/net.go:183 +0x45 io.ReadAll({0xaa2c20, 0xc011e78848}) /usr/local/go/src/io/io.go:645 +0xfe io/ioutil.ReadAll(...) /usr/local/go/src/io/ioutil/ioutil.go:27 name/internal/whois.(Client).rawQuery(0xc01d25e1e0, {0xc002dd9ad0, 0x8}, {0xc01caf5965, 0xe}, {0x9a907f, 0x2}) /home/name/internal/whois/li.go:203 +0x5f4 name/internal/whois.(*Client).Whois(0x9b58f8?, {0xc002dd9ad0, 0x8}, {0xc00b7f7758, 0x1, 0x3?}) /home/name/internal/whois/li.go:156 +0x4ab name/internal/whois.ProxyWhois({0x9b58f8?, 0x16?}, {0xc002dd9ad0, 0x8}, {0x9bdc4b, 0x16}) /home/name/internal/whois/li.go:105 +0x68 name/internal/whois.GetBulkNormalWhois({0xc002dd9ad0?, 0x8}) /home/name/internal/whois/whois.go:501 +0x4b0 name/internal/whoislookup.getwhoisme({0xc002dd9ad0, 0x8}, 0x1, 0xc00061e018) /home/name/internal/whoislookup/mywhois.go:232 +0x4d name/internal/whoislookup.getInfome({0xc002dd9ad0?, 0x0?}, 0x9af93d?, 0xd?, 0x9bd0c9?, 0x15?, 0x9afb38?) /home/name/internal/whoislookup/mywhois.go:269 +0x65 created by name/internal/whoislookup.collectme /home/name/internal/whoislookup/mywhois.go:296 +0x26d

goroutine 907686 [runnable]: internal/poll.runtime_pollWait(0x7fb8d8d13e18, 0x72) /usr/local/go/src/runtime/netpoll.go:302 +0x89 internal/poll.(pollDesc).wait(0xc01fe96100?, 0xc01df0e200?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32 internal/poll.(pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:88 internal/poll.(FD).Read(0xc01fe96100, {0xc01df0e200, 0x200, 0x200}) /usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a net.(netFD).Read(0xc01fe96100, {0xc01df0e200?, 0xc0138fb458?, 0x44be12?}) /usr/local/go/src/net/fd_posix.go:55 +0x29 net.(conn).Read(0xc0118fe6b8, {0xc01df0e200?, 0xe33d8ccdaf?, 0xda1160?}) /usr/local/go/src/net/net.go:183 +0x45 io.ReadAll({0xaa2c20, 0xc0118fe6b8}) /usr/local/go/src/io/io.go:645 +0xfe io/ioutil.ReadAll(...) /usr/local/go/src/io/ioutil/ioutil.go:27 name/internal/whois.(Client).rawQuery(0xc01edbd540, {0xc002dd83a0, 0xd}, {0xc015d65c6a, 0xe}, {0x9a907f, 0x2}) /home/name/internal/whois/li.go:203 +0x5f4 name/internal/whois.(*Client).Whois(0x9bae92?, {0xc002dd83a0, 0xd}, {0xc00b7fb758, 0x1, 0x3?}) /home/name/internal/whois/li.go:156 +0x4ab name/internal/whois.ProxyWhois({0x9bae92?, 0x16?}, {0xc002dd83a0, 0xd}, {0x9bdc4b, 0x16}) /home/name/internal/whois/li.go:105 +0x68 name/internal/whois.GetBulkNormalWhois({0xc002dd83a0?, 0xd}) /home/name/internal/whois/whois.go:501 +0x4b0 name/internal/whoislookup.getwhoisme({0xc002dd83a0, 0xd}, 0x1, 0xc00061e018) /home/name/internal/whoislookup/mywhois.go:232 +0x4d name/internal/whoislookup.getInfome({0xc002dd83a0?, 0x3c3c3c205a32313a?}, 0x726f6d20726f4620?, 0x6d726f666e692065?, 0x6e6f206e6f697461?, 0x20?, 0x6f63207375746174?) /home/name/internal/whoislookup/mywhois.go:269 +0x65 created by name/internal/whoislookup.collectme /home/name/internal/whoislookup/mywhois.go:296 +0x26d

buffer, err := ioutil.ReadAll(conn)
namecost commented 1 year ago

190 行

likexian commented 1 year ago

日志不完整,以上日志没问题

namecost commented 1 year ago

这个是不是要加一个判断,conn这个。 我测试了500w的数据查询,并发200. 当进行到100多万的时候,出现了这个。然后程序中断。 自己设置了代理。 func NewPorxyClient(address string) *Client { proxyDialer, err := proxy.SOCKS5("tcp", address, nil, proxy.Direct) if err != nil { fmt.Println(err) } return &Client{ dialer: proxyDialer.(proxy.Dialer), timeout: defaultTimeout, } }

func ProxyWhois(proxy, domain, servers string) (result string, err error) { return NewPorxyClient(proxy).Whois(domain, servers) }

likexian commented 1 year ago

单看代码没问题,你往我邮箱发下完整的日志吧