go-rod / rod

A Chrome DevTools Protocol driver for web automation and scraping.
https://go-rod.github.io
MIT License
5k stars 328 forks source link

Failed to deserialize params.body - BINDINGS: binary value expected at position 61 #1067

Closed hktalent closed 3 weeks ago

hktalent commented 4 weeks ago

Rod Version:v0.114.8

hj.ContinueRequest(&proto.FetchContinueRequest{})
image

other:

HijackRequests error {-32000 Fetch domain is not enabled }
github-actions[bot] commented 4 weeks ago

Please add a valid Rod Version: v0.0.0 to your issue. Current version is v0.116.0

Please fix the format of your markdown:

2 MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```"]
2 MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
6:1 MD033/no-inline-html Inline HTML [Element: img]
10 MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]

generated by check-issue

hktalent commented 3 weeks ago

尝试不加载这些资源,都将导致无法继续后续的,也不知道为什么

ext := filepath.Ext(ctx.Request.URL().Path)
        szT := util.Any2Str(ctx.Request.Req().Header["Content-Type"]) // stylesheet,font, image
        // 阻止请求 strings.Contains(szT, "stylesheet") ||
        if strings.Contains(szT, "stylesheet") || strings.Contains(szT, "font") || strings.Contains(szT, "image") || ext == ".png" || ext == ".jpg" || ext == ".jpeg" || ext == ".gif" || ext == ".css" || ext == ".woff" || ext == ".woff2" || ext == ".ttf" || ext == ".eot" {
            //ctx.Skip = true // ctx.Request.Req().Header.Set("My-Header", "test")
            //ctx.Response.Fail(proto.NetworkErrorReasonAborted)
            return false
        }
ysmood commented 3 weeks ago

抱歉,我看不太懂你描述

hktalent commented 3 weeks ago

1、MustAdd 添加了拦截器,我因为是做爬虫,所以想阻止图片、字体、css文件的加载,始终没有成功 2、我希望在页面加载完了、添加的拦截事件都运行完了得到一个事件,及时关闭page,似乎爬虫处理的页面在 page.MustNavigate(szUrl.Url).MustWaitLoad() 之后立刻关闭page的话,很多拦截器就无法得到运行 我当前是基于延时70秒,但是这不太友好

您可以尝试下爬虫:https://www.alipan.com/s/4iQjiU9KUmG 1、抓取"/adrive/v2/file/list_by_share" 的数据 3、阻止图片、字体、css文件的加载

我不确定:

wait := page.MustWaitNavigation()
    page.MustNavigate(szUrl.Url).MustWaitLoad()
    wait()
我不确定这里关闭page是否合适
hktalent commented 3 weeks ago

这个错误是由于爬虫处理过程中这文件返回的是图片,然后就这样了

router.MustAdd(k1, func(hj *rod.Hijack) {
                    hj.OnError = func(err error) {
                        log.Println("HijackRequests error", err, hj.Request.URL().String())
                    }
......

HijackRequests error {-32602 Invalid parameters Failed to deserialize params.body - BINDINGS: binary value expected at position 64} https://cn-beijing-data.aliyundrive.net/pxcIoOce%2F951346711%2F65cf1bc23d2e20b1bf52475a955ec4e247b8049a%2F65cf1bc2c77c291d35294a6db78d47e7ee32c861?security-token=CAISvgJ1q6Ft5B2yfSjIr5bDfNXu3qxT5fOJW1fej0wFb8tLqZH5izz2IHhMf3NpBOkZvvQ1lGlU6%2Fcalq5rR4QAXlDfNRjYXxzCqFHPWZHInuDox55m4cTXNAr%2BIhr%2F29CoEIedZdjBe%2FCrRknZnytou9XTfimjWFrXWv%2Fgy%2BQQDLItUxK%2FcCBNCfpPOwJms7V6D3bKMuu3OROY6Qi5TmgQ41Uh1jgjtPzkkpfFtkGF1GeXkLFF%2B97DRbG%2FdNRpMZtFVNO44fd7bKKp0lQLs0ARrv4r1fMUqW2X543AUgFLhy2KKMPY99xpFgh9a7j0iCbSGyUu%2FhcRm5sw9%2Byfo34lVYne4%2FFR30p3LuHwufJ7FxfIREfquk63pvSlHLcLPe0Kjzzleo2k1XRPVFF%2B535IaHXuToXDnvSihKwGRvXtuMkagAFeAve5ns3ex0wAOSBalZvD0klFJxbg3effir9a7vEkZDzJOXxAlR7yiQXj1lcSxPnrCbdziYangfTPvr46KuhWuTkdD5%2FUwDg56Lz6r2Wg3faIFmc0r%2FWequSF52z0R8QmwZKpkPR2Sfd4RTWk4DjG5fjgsLVIVYs8GjhiY5uBySAA&x-oss-access-key-id=STS.NUv7oZ3srR1bYqokLPcDgFTRi&x-oss-expires=1717268108&x-oss-process=video%2Fsnapshot%2Ct_1000%2Cf_jpg%2Car_auto%2Cw_256&x-oss-signature=T%2B6BUXZy%2BJ4PhXJYxxsl3o9UipOJ%2BwaGn1SfzVganmw%3D&x-oss-signature-version=OSS2