go-rod / rod

A Chrome DevTools Protocol driver for web automation and scraping.
https://go-rod.github.io
MIT License
5k stars 328 forks source link

Get all javascript or css resources of the same-origin #1037

Closed and0x00 closed 2 months ago

and0x00 commented 2 months ago

Rod Version: v0.114.8

Hi. Is there any way to list the JavaScript and CSS resources loaded by the browser when accessing the site using the frames as shown in the print or the sources?

Screenshot 2024-04-12 at 22 39 32

I tried it this way, but only the URLs of resources from other domains are printed. I would like it to also show the local resources.


    var e proto.NetworkResponseReceived
    wait := page.WaitEvent(&e)
    page.MustNavigate(URL).MustWaitLoad().MustWaitDOMStable().MustWaitStable()
    wait()

    xhrURLs := make([]string, 0)
    go page.EachEvent(func(e *proto.NetworkRequestWillBeSent) {
        if e.DocumentURL != "" {
            if e.Type == proto.NetworkResourceTypeXHR || e.Type == proto.NetworkResourceTypeFetch {
                fmt.Printf("%v\n", e.Request.URL)
                // xhrURLs = append(xhrURLs, e.Request.URL)
            }
            if e.Type == proto.NetworkResourceTypeScript || e.Type == proto.NetworkResourceTypeStylesheet {
                fmt.Printf("%v\n", e.Request.URL)
            }

            if e.Type == proto.NetworkResourceTypeDocument {
                fmt.Printf("%v\n", e.Request.URL)
            }
        }
    })()
Fly-Playgroud commented 2 months ago

You must listen to the event before navigating

and0x00 commented 2 months ago

That worked!!! Thank you very much!!!

github-actions[bot] commented 2 months ago

Please add a valid Rod Version: v0.0.0 to your issue. Current version is v0.115.0

Please fix the format of your markdown:

3:163 MD009/no-trailing-spaces Trailing spaces [Expected: 0 or 2; Actual: 1]
7:1 MD033/no-inline-html Inline HTML [Element: img]

generated by check-issue