search

go-rod / rod

A Chrome DevTools Protocol driver for web automation and scraping.
https://go-rod.github.io
MIT License
5.43k stars 357 forks source link

"Page" doesn't work in vivaldi browser. #1143

Open aca opened 1 week ago

aca commented 1 week ago

Another chromium based browser vivaldi doesn't seem to work with rod. I haven't tested all, but playwright-py can navigate pages. Is there a way to use vivaldi with go-rod?

package main

import (
    "encoding/json"
    "log"
    "net/http"

    "github.com/go-rod/rod"
)

// run browser (macos)
// open -a Vivaldi --args --remote-debugging-port=9222

func main() {
    wsurl := getWebsocketDebuggerUrl("http://localhost:9222")
    log.Println(wsurl)
    rod.New().ControlURL(wsurl).MustConnect().MustPage("https://github.com")
}

type VersionResponse struct {
    WebSocketDebuggerUrl string `json:"webSocketDebuggerUrl"`
}

func getWebsocketDebuggerUrl(url string) string {
    resp, err := http.Get(url + "/json/version")
    if err != nil {
        log.Fatalf("Failed to make request: %v", err)
    }
    defer resp.Body.Close()
    var versionResp VersionResponse
    err = json.NewDecoder(resp.Body).Decode(&versionResp)
    if err != nil {
        log.Fatalf("Failed to decode JSON: %v", err)
    }
    return versionResp.WebSocketDebuggerUrl
}

2024/11/13 13:27:59 ws://localhost:9222/devtools/browser/0244adc3-594d-42df-9f53-dcb40a9880fe
panic: EOF

goroutine 1 [running]:
github.com/go-rod/rod/lib/utils.init.func2({0x102de9c40?, 0x10311b670?})
        /Users/x/pkg/mod/github.com/go-rod/rod@v0.116.2/lib/utils/utils.go:68 +0x24
main.main.New.(*Browser).WithPanic.genE.func2({0x14000288590?, 0x102a5bf9d?, 0x12?})
        /Users/x/pkg/mod/github.com/go-rod/rod@v0.116.2/must.go:36 +0x70
github.com/go-rod/rod.(*Browser).MustPage(0x14000224360, {0x140000dff08?, 0x2?, 0x140000dff18?})
        /Users/x/pkg/mod/github.com/go-rod/rod@v0.116.2/must.go:71 +0xec
main.main()
        /Users/x/src/scratch/2024/11/20241113T131333/rod/main.go:17 +0x2fc
exit status 2
github-actions[bot] commented 1 week ago

Please add a valid Rod Version: v0.0.0 to your issue. Current version is v0.116.2

Please fix the format of your markdown:

3 MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
40 MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```"]
41 MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```"]
41 MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]

generated by check-issue

ysmood commented 1 week ago

better to enable cdp log to see what's goes wrong:

go run main.go -rod=cdp
aca commented 1 week ago

Thank you for the reply. Now I could clarify the problem... But still not sure what to do.. Removing page.Emulate, page.EnableDomain seems to work.. I can bypass page.Emulate with NoDefaultDevice but not sure how to handle page.EnableDomain

https://github.com/go-rod/rod/blob/0096e531a49c82f272f5661eabe755d2e7c1eb61/browser.go?plain=1#L300-L313

vivaldi 

[cdp] 2024/11/14 15:35:12 => #1 @00000000 Target.setDiscoverTargets {"discover":true}
[cdp] 2024/11/14 15:35:12 <- @00000000 Target.targetCreated {"targetInfo":{"targetId":"01AF8B2E986E8711A0A5968E8C317EBC","type":"app","title":"Vivaldi","url":"chrome-extension://mpognobbkildjkofajifpdfhcoklimli/window.html","attached":false,"canAccessOpener":false,"browserContextId":"C7C126F491A21011F420E77F7DA0E91D"}}
[cdp] 2024/11/14 15:35:12 <- @00000000 Target.targetCreated {"targetInfo":{"targetId":"4121C397296CA26802F5A9812BF50B7F","type":"page","title":"chrome://vivaldi-webui/startpage","url":"chrome://vivaldi-webui/startpage","attached":false,"canAccessOpener":false,"browserContextId":"C7C126F491A21011F420E77F7DA0E91D"}}
[cdp] 2024/11/14 15:35:12 <- @00000000 Target.targetCreated {"targetInfo":{"targetId":"41B27E34166467161F2612DE60AA934E","type":"app","title":"Vivaldi","url":"chrome-extension://mpognobbkildjkofajifpdfhcoklimli/main.html","attached":false,"canAccessOpener":false,"browserContextId":"C7C126F491A21011F420E77F7DA0E91D"}}
[cdp] 2024/11/14 15:35:12 <- @00000000 Target.targetCreated {"targetInfo":{"targetId":"AF7557DE492BEF0EF9A1BC36F3B9F565","type":"page","title":"chrome://vivaldi-webui/startpage?section=Speed-dials&background-color=#f7f7f7","url":"chrome://vivaldi-webui/startpage?section=Speed-dials&background-color=#f7f7f7","attached":false,"canAccessOpener":false,"browserContextId":"C7C126F491A21011F420E77F7DA0E91D"}}
[cdp] 2024/11/14 15:35:12 <- @00000000 Target.targetCreated {"targetInfo":{"targetId":"BE037F91CBA7400E1838E083D027118F","type":"service_worker","title":"Service Worker chrome-extension://mpognobbkildjkofajifpdfhcoklimli/background-service-worker-e5407a8aaef5992b3839cd54d3629909.js","url":"chrome-extension://mpognobbkildjkofajifpdfhcoklimli/background-service-worker-e5407a8aaef5992b3839cd54d3629909.js","attached":false,"canAccessOpener":false,"browserContextId":"C7C126F491A21011F420E77F7DA0E91D"}}
[cdp] 2024/11/14 15:35:12 <- @00000000 Target.targetCreated {"targetInfo":{"targetId":"D030F4EF4ED62DF77B3361A495257B74","type":"shared_worker","title":"","url":"chrome-extension://mpognobbkildjkofajifpdfhcoklimli/bundle-mailreader-worker.js","attached":false,"canAccessOpener":false,"browserContextId":"C7C126F491A21011F420E77F7DA0E91D"}}
[cdp] 2024/11/14 15:35:12 <= #1 {}
[cdp] 2024/11/14 15:35:12 => #2 @00000000 Target.createTarget {"url":"about:blank"}
[cdp] 2024/11/14 15:35:12 <- @00000000 Target.targetCreated {"targetInfo":{"targetId":"0F0A4E8E4C2BA654083FA67D4E1E8AD3","type":"page","title":"","url":"about:blank","attached":false,"canAccessOpener":false,"browserContextId":"C7C126F491A21011F420E77F7DA0E91D"}}
[cdp] 2024/11/14 15:35:12 <= #2 {"targetId":"0F0A4E8E4C2BA654083FA67D4E1E8AD3"}
[cdp] 2024/11/14 15:35:12 => #3 @00000000 Target.attachToTarget {"targetId":"0F0A4E8E4C2BA654083FA67D4E1E8AD3","flatten":true}
[cdp] 2024/11/14 15:35:12 <- @00000000 Target.targetInfoChanged {"targetInfo":{"targetId":"0F0A4E8E4C2BA654083FA67D4E1E8AD3","type":"page","title":"","url":"","attached":true,"canAccessOpener":false,"browserContextId":"C7C126F491A21011F420E77F7DA0E91D"}}
[cdp] 2024/11/14 15:35:12 <- @00000000 Target.attachedToTarget {"sessionId":"A2072B73FBB37B6840265832642041EA","targetInfo":{"targetId":"0F0A4E8E4C2BA654083FA67D4E1E8AD3","type":"page","title":"","url":"","attached":true,"canAccessOpener":false,"browserContextId":"C7C126F491A21011F420E77F7DA0E91D"},"waitingForDebugger":false}
[cdp] 2024/11/14 15:35:12 <= #3 {"sessionId":"A2072B73FBB37B6840265832642041EA"}
[cdp] 2024/11/14 15:35:12 => #4 @A2072B73 Emulation.setDeviceMetricsOverride {"width":1280,"height":800,"deviceScaleFactor":1,"mobile":false,"screenOrientation":{"type":"landscapePrimary","angle":90}}
[cdp] 2024/11/14 15:35:12 => #5 @00000000 Target.closeTarget {"targetId":"0F0A4E8E4C2BA654083FA67D4E1E8AD3"}
panic: EOF

chromium (works fine)

2024/11/14 15:35:37 ws://localhost:9222/devtools/browser/c96f75ca-a4e1-4c2d-bc1a-27970346d655
[cdp] 2024/11/14 15:35:37 => #1 @00000000 Target.setDiscoverTargets {"discover":true}
[cdp] 2024/11/14 15:35:37 <- @00000000 Target.targetCreated {"targetInfo":{"targetId":"22DA3A5BB30E9CE5293AA35C8E1EBF0C","type":"iframe","title":"https://ogs.google.com/u/0/widget/app?eom=1&awwd=1&origin=chrome-untrusted%3A%2F%2Fnew-tab-page&origin=chrome%3A%2F%2Fnew-tab-page&cn=app&pid=1&spid=243&hl=en","url":"https://ogs.google.com/u/0/widget/app?eom=1&awwd=1&origin=chrome-untrusted%3A%2F%2Fnew-tab-page&origin=chrome%3A%2F%2Fnew-tab-page&cn=app&pid=1&spid=243&hl=en","attached":false,"canAccessOpener":false,"browserContextId":"B6A2CF4EEF2CEB9A6EC1D215EF24CDD9"}}
[cdp] 2024/11/14 15:35:37 <- @00000000 Target.targetCreated {"targetInfo":{"targetId":"31D9A7D1F941C0850C3680DF0507FE46","type":"service_worker","title":"Service Worker chrome-extension://gfbliohnnapiefjpjlpjnehglfpaknnc/background.js","url":"chrome-extension://gfbliohnnapiefjpjlpjnehglfpaknnc/background.js","attached":false,"canAccessOpener":false,"browserContextId":"B6A2CF4EEF2CEB9A6EC1D215EF24CDD9"}}
[cdp] 2024/11/14 15:35:37 <- @00000000 Target.targetCreated {"targetInfo":{"targetId":"48562D85CDE17BA86ED7EC829140F665","type":"page","title":"New Tab","url":"chrome://newtab/","attached":false,"canAccessOpener":false,"browserContextId":"B6A2CF4EEF2CEB9A6EC1D215EF24CDD9"}}
[cdp] 2024/11/14 15:35:37 <- @00000000 Target.targetCreated {"targetInfo":{"targetId":"7764C7FF8265C279B56253B3CB27DE58","type":"service_worker","title":"Service Worker chrome-extension://ddkjiahejlhfcafbddmgiahcphecmpfh/js/background.js","url":"chrome-extension://ddkjiahejlhfcafbddmgiahcphecmpfh/js/background.js","attached":false,"canAccessOpener":false,"browserContextId":"B6A2CF4EEF2CEB9A6EC1D215EF24CDD9"}}
[cdp] 2024/11/14 15:35:37 <- @00000000 Target.targetCreated {"targetInfo":{"targetId":"AAE57D7C8677B63D6EF1BB45911DA154","type":"iframe","title":"chrome-untrusted://new-tab-page/one-google-bar?paramsencoded=","url":"chrome-untrusted://new-tab-page/one-google-bar?paramsencoded=","attached":false,"canAccessOpener":false,"browserContextId":"B6A2CF4EEF2CEB9A6EC1D215EF24CDD9"}}
[cdp] 2024/11/14 15:35:37 <= #1 {}
[cdp] 2024/11/14 15:35:37 => #2 @00000000 Target.createTarget {"url":"about:blank"}
[cdp] 2024/11/14 15:35:37 <- @00000000 Target.targetCreated {"targetInfo":{"targetId":"E2CF607F0ADF20FB5A9A66578BD7934F","type":"page","title":"","url":"about:blank","attached":false,"canAccessOpener":false,"browserContextId":"B6A2CF4EEF2CEB9A6EC1D215EF24CDD9"}}
[cdp] 2024/11/14 15:35:37 <= #2 {"targetId":"E2CF607F0ADF20FB5A9A66578BD7934F"}
[cdp] 2024/11/14 15:35:37 => #3 @00000000 Target.attachToTarget {"targetId":"E2CF607F0ADF20FB5A9A66578BD7934F","flatten":true}
[cdp] 2024/11/14 15:35:37 <- @00000000 Target.targetInfoChanged {"targetInfo":{"targetId":"E2CF607F0ADF20FB5A9A66578BD7934F","type":"page","title":"about:blank","url":"about:blank","attached":false,"canAccessOpener":false,"browserContextId":"B6A2CF4EEF2CEB9A6EC1D215EF24CDD9"}}
[cdp] 2024/11/14 15:35:37 <- @00000000 Target.targetInfoChanged {"targetInfo":{"targetId":"E2CF607F0ADF20FB5A9A66578BD7934F","type":"page","title":"about:blank","url":"about:blank","attached":true,"canAccessOpener":false,"browserContextId":"B6A2CF4EEF2CEB9A6EC1D215EF24CDD9"}}
[cdp] 2024/11/14 15:35:37 <- @00000000 Target.attachedToTarget {"sessionId":"5D6148FF23D1DF538AD724A68FAC7031","targetInfo":{"targetId":"E2CF607F0ADF20FB5A9A66578BD7934F","type":"page","title":"about:blank","url":"about:blank","attached":true,"canAccessOpener":false,"browserContextId":"B6A2CF4EEF2CEB9A6EC1D215EF24CDD9"},"waitingForDebugger":false}
[cdp] 2024/11/14 15:35:37 <= #3 {"sessionId":"5D6148FF23D1DF538AD724A68FAC7031"}
[cdp] 2024/11/14 15:35:37 => #4 @5D6148FF Emulation.setDeviceMetricsOverride {"width":1280,"height":800,"deviceScaleFactor":1,"mobile":false,"screenOrientation":{"type":"landscapePrimary","angle":90}}
[cdp] 2024/11/14 15:35:37 <= #4 {}
[cdp] 2024/11/14 15:35:37 => #5 @5D6148FF Emulation.setTouchEmulationEnabled {"enabled":false,"maxTouchPoints":5}
[cdp] 2024/11/14 15:35:37 <= #5 {}
[cdp] 2024/11/14 15:35:37 => #6 @5D6148FF Network.setUserAgentOverride {"userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36","acceptLanguage":"en"}
[cdp] 2024/11/14 15:35:37 <= #6 {}
[cdp] 2024/11/14 15:35:37 => #7 @5D6148FF Page.enable {}
[cdp] 2024/11/14 15:35:37 <= #7 {}
[cdp] 2024/11/14 15:35:37 => #8 @5D6148FF Page.stopLoading {}
[cdp] 2024/11/14 15:35:37 <= #8 {}
[cdp] 2024/11/14 15:35:37 => #9 @5D6148FF Page.navigate {"url":"https://google.com"}
[cdp] 2024/11/14 15:35:37 <- @5D6148FF Page.frameStartedLoading {"frameId":"E2CF607F0ADF20FB5A9A66578BD7934F"}
[cdp] 2024/11/14 15:35:37 <= #9 {"frameId":"E2CF607F0ADF20FB5A9A66578BD7934F","loaderId":"9DD56BFEE2F1AB98D3FF41E216F2CE99"}
ysmood commented 1 week ago

I think it's an issue of vivaldi browser, rod can't do much about it, I don't have enough resource to fix it.