go-rod / rod

A Chrome DevTools Protocol driver for web automation and scraping.
https://go-rod.github.io
MIT License
5k stars 328 forks source link

Add an initEvents public function with a Browser structure #1046

Open sxmxta opened 2 months ago

sxmxta commented 2 months ago

Hello I am the author of the Energy open-source framework This framework was developed by Go based on CEF Someone requested to use Energy to create an automated tool for visual graphic control, and gave some suggestions, including Rod Afterwards, I researched rod and found that it is possible to use the CEF API to directly send devtools protocol methods without the need for websocket or opening the devtools remote debugging port. Later, I tried to modify the message sending and receiving events and processing of rod, and changed it to CEF API sending messages and CEF API callback events receiving messages. This is completely feasible. And it won't invade rod But after modification, it was found that the simplest solution is to provide a public InitEvents function in the borwser structure of rod, which is currently non-public

browser.go Before modification

func (b *Browser) initEvents() {
    ctx, cancel := context.WithCancel(b.ctx)
    b.event = goob.New(ctx)
    event := b.client.Event()

    go func() {
        defer cancel()
        for e := range event {
            b.event.Publish(&Message{
                SessionID: proto.TargetSessionID(e.SessionID),
                Method:    e.Method,
                lock:      &sync.Mutex{},
                data:      e.Params,
            })
        }
    }()
}

After modification

func (b *Browser) InitEvents(){
    if b.event == nil {
        b.initEvents()
    }
}

func (b *Browser) initEvents() {
    ctx, cancel := context.WithCancel(b.ctx)
    b.event = goob.New(ctx)
    event := b.client.Event()

    go func() {
        defer cancel()
        for e := range event {
            b.event.Publish(&Message{
                SessionID: proto.TargetSessionID(e.SessionID),
                Method:    e.Method,
                lock:      &sync.Mutex{},
                data:      e.Params,
            })
        }
    }()
}

After creation, fully utilize the functions provided by rod

github-actions[bot] commented 2 months ago

Please add a valid Rod Version: v0.0.0 to your issue. Current version is v0.115.0

generated by check-issue

sxmxta commented 2 months ago

Try using browser Connect () initializes InitEvents, but it blocks the UI thread.

There won't be any problem with InitEvents directly

ysmood commented 2 months ago

How about use the cdp directly? Look at the code client.Event():

https://github.com/go-rod/rod/blob/3557c232027e27173c8d09fd08579bfe057e5e59/lib/cdp/example_test.go#L14-L44

sxmxta commented 2 months ago

How about use the cdp directly? Look at the code client.Event():

https://github.com/go-rod/rod/blob/3557c232027e27173c8d09fd08579bfe057e5e59/lib/cdp/example_test.go#L14-L44

Hello, this method is not suitable. I have switched to another method, using "go Browser.Connect()" instead of initEvents().

Because it directly passes a []byte, I need to know the id and method, and I don't want to deserialize again to obtain them. I am using the Chromium().ExecuteDevToolsMethod(messageId int32, method string, dictionaryValue *ICefDictionaryValue) function, so I have replaced it with "go Browser.Connect()". Let's try this approach for now.

func (m *Energy) CreateBrowser() {
    if !m.created {
        m.created = true
        go m.rodBrowser.Connect()
        //m.rodBrowser.InitEvents()
        if m.window != nil {
            // window
            if m.window.IsLCL() {
                cef.RunOnMainThread(func() {
                    m.window.Show()
                })
            } else {
                m.window.Show()
            }
        } else if m.chromiumBrowser != nil {
            m.chromiumBrowser.CreateBrowser()
        }
    }
}

// Call a method and wait for its response.
func (m *Energy) Call(ctx context.Context, sessionID, method string, params interface{}) ([]byte, error) {
    req := &cdp.Request{
        ID:        int(atomic.AddUint64(&m.count, 1)),
        SessionID: sessionID,
        Method:    method,
        Params:    params,
    }
    m.logger.Println(req)
    data, err := json.Marshal(params)
    utils.E(err)
    done := make(chan Result)
    once := sync.Once{}
    m.pending.Store(req.ID, func(res Result) {
        once.Do(func() {
            select {
            case <-ctx.Done():
            case done <- res:
            }
        })
    })
    defer m.pending.Delete(req.ID)
    //m.logger.Println("send-data:", string(data))
    //m.chromium.SendDevToolsMessage(string(data))// Linux cannot be used
    dict := JSONParse(data)
    m.ChromiumBrowser().Chromium().ExecuteDevToolsMethod(int32(req.ID), req.Method, dict)
    defer dict.Free()
    select {
    case <-ctx.Done():
        return nil, ctx.Err()
    case res := <-done:
        return res.Msg, res.Err
    }
}
ysmood commented 1 month ago

You don't have to marshal it manually, you can use the proto lib:

// Package main ...
package main

import (
    "fmt"

    "github.com/go-rod/rod/lib/cdp"
    "github.com/go-rod/rod/lib/launcher"
    "github.com/go-rod/rod/lib/proto"
    "github.com/go-rod/rod/lib/utils"
)

func main() {
    // launch a browser
    url := launcher.New().MustLaunch()

    // create a controller
    client := cdp.New().Start(cdp.MustConnectWS(url))

    go func() {
        for range client.Event() {
            // you must consume the events
            utils.Noop()
        }
    }()

    res, err := proto.TargetCreateTarget{URL: "http://test.com"}.Call(client)
    if err != nil {
        panic(err)
    }

    fmt.Println(res.TargetID)

    _ = proto.BrowserClose{}.Call(client)
}