go-rod / rod

A Chrome DevTools Protocol driver for web automation and scraping.
https://go-rod.github.io
MIT License
5.37k stars 353 forks source link

go browser.MustHandleAuth(username, proxyPassword)() doesn't work in AWS Lambda #1112

Open SrikanthKumarC opened 2 months ago

SrikanthKumarC commented 2 months ago

Rod Version: v0.113.0 I'm using a customized verison of https://github.com/YoungiiJC/go-rod-aws-lambda

Please check the gist here: https://gist.github.com/SrikanthKumarC/1f65571364cfebc6a002562464a12d9f

Error response when ran with sam local invoke GetPageHTMLFunction -e event.json:

goroutine 31 [running]: github.com/go-rod/rod/lib/utils.glob..func2({0x91df20?, 0xc0000965a0?}) /go/pkg/mod/github.com/go-rod/rod@v0.113.0/lib/utils/utils.go:68 +0x25 github.com/go-rod/rod.genE.func1({0xc0002072d0?, 0xade800000003be70?, 0x1b8000000?}) /go/pkg/mod/github.com/go-rod/rod@v0.113.0/must.go:36 +0x5a github.com/go-rod/rod.(*Browser).MustHandleAuth.func1() /go/pkg/mod/github.com/go-rod/rod@v0.113.0/must.go:92 +0x93 created by main.getPageHTML.func1 /app/rod.go:127 +0x217 09 Sep 2024 10:38:29,677 [ERROR] (rapid) Invoke failed error=Runtime exited with error: exit status 2 InvokeID=c52cf075-9ac7-4384-a314-6dee9a1e309f 09 Sep 2024 10:38:29,678 [ERROR] (rapid) Invoke DONE failed: Runtime.ExitError {"errorType": "Runtime.ExitError", "errorMessage": "RequestId: c52cf075-9ac7-4384-a314-6dee9a1e309f Error: Runtime exited with error: exit status 2"}

github-actions[bot] commented 2 months ago

Please fix the format of your markdown:

2:81 MD009/no-trailing-spaces Trailing spaces [Expected: 0 or 2; Actual: 1]
6:83 MD009/no-trailing-spaces Trailing spaces [Expected: 0 or 2; Actual: 1]
8:1 MD038/no-space-in-code Spaces inside code span elements [Context: "`"]

generated by check-issue

SrikanthKumarC commented 2 months ago

Removing this code makes it work, but I still want to use it. I checked other issues and changed ordering but to no avail.

            router := page.HijackRequests()
            resources := []proto.NetworkResourceType{
            proto.NetworkResourceTypeFont,
            proto.NetworkResourceTypeImage,
            proto.NetworkResourceTypeMedia,
            proto.NetworkResourceTypeStylesheet,
            proto.NetworkResourceTypeWebSocket, // we don't need websockets to fetch html
                 }

        router.MustAdd("*", func(ctx *rod.Hijack) {
            if slice.Contains(resources, ctx.Request.Type()) {
                ctx.Response.Fail(proto.NetworkErrorReasonBlockedByClient)
                return
            }

            ctx.ContinueRequest(&proto.FetchContinueRequest{})
        })

        go router.Run()