gospider007 / gospider

🚀Gospider is a powerful Golang web crawler that includes all the necessary libraries for transitioning from Python to Golang. It provides a fast and seamless transition for Python web crawlers to Golang.
GNU Lesser General Public License v3.0
91 stars 22 forks source link

自实现代理协议与go代理协议的实现有差异 #3

Closed vodves-vodves closed 1 year ago

vodves-vodves commented 1 year ago

Doesn't work with resident proxies? It's just that if I run with regular proxies, everything works, but not with resident proxies.)

gospider007 commented 1 year ago

可以提供报错信息吗,或者代码示例

vodves-vodves commented 1 year ago
reqCli, err := requests.NewClient(nil)
    if err != nil {
        fmt.Println(err)
    }
    ja3Str := "772,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-45-11-10-27-18-17513-65281-23-5-16-43-35-51-13,29-23-24,0"
    Ja3Spec, err := ja3.CreateSpecWithStr(ja3Str)
    if err != nil {
        fmt.Println(err)
    }

    response, err := reqCli.Request(nil, "get", "https://examplesite.com/", requests.RequestOption{Ja3Spec: Ja3Spec, Proxy: "http://login:pass@46.8.202.78:7956"}) 
    if err != nil {
        fmt.Println(err)
        return
    }
    log.Print(response.Text())

Everything works with the proxies above

And with this resident proxy does not work: http://login:pass@s1.reproxy.network:19000 (I removed the authorization information)

Get "https://examplesite.com/": write tcp ip:port->94.131.7.1:19000: wsasend: An existing connection was forcibly closed by the remote host.
gospider007 commented 1 year ago

从报错信息来看,应该是远程主机主动关闭了连接导致的,事实上客户端已经与代理服务器建立连接了,代理服务器因为未知原因主动关闭了连接。 我有个疑问,这个代理在其他的客户端上是否能正常使用呢,如果在其他的客户端上也不能使用,是否意味着这个代理有问题

vodves-vodves commented 1 year ago

Everything works fine with the standard golang client

gospider007 commented 1 year ago

我大概已经知道出现这个问题的原因,这个问题我已经尝试在最新版修复它了,你现在可以更新模块,测试下这个问题是否被修复了

vodves-vodves commented 1 year ago

Nothing has changed) I can give you one resident proxy so you can test

gospider007 commented 1 year ago

好的,请将代理发送到 2216403312@qq.com 这个邮箱

gospider007 commented 1 year ago

当我的ip来源是亚洲地区时,使用你提供的代理出现如下报错:

Get "https://examplesite.com/": read tcp 192.168.1.239:51536->94.131.7.1:19001: wsarecv: An existing connection was forcibly closed by the remote host.

当我的ip来源是美国地区时,使用你提供的代理出现如下报错:

 Get "https://examplesite.com/": Unauthorized

我认为你提供的代理限制了使用者的ip地区。并且你给我提供的用户名密码存在错误,导致从而出现Unauthorized,以上问题当我使用golang 标准客户端也会出现。