m1k1o / neko

A self hosted virtual browser that runs in docker and uses WebRTC.
https://neko.m1k1o.net/
Apache License 2.0
5.98k stars 449 forks source link

Implicit control multiple modes #163

Open gbrian opened 2 years ago

gbrian commented 2 years ago

When enabling "implicit control" mouse events are mixed on remote causing other members mouse events interfeer with current user action (like drag, hightlight,...)

What if we move to a click and control mode? In this way user must "click" first to take control. Working on a demo

m1k1o commented 2 years ago

We could have multiple modes of NEKO_IMPLICIT_CONTROL:

And current implementation should deny control takeover when dragging something, or maybe actively moving or when typing.

gbrian commented 11 months ago

Back to this topic as is a bit annoying, sharing a basic approach to keep the beauty of multi-control but avoiding the mess when dragging on pointing on the screen when other is on control.

Add hasControl to SessionManager so we know which host's events reac to

type SessionManager struct {
    mu            sync.Mutex
    logger        zerolog.Logger
    host          string
    capture       types.CaptureManager
    members       map[string]*Session
    eventsChannel chan types.SessionEvent
    // TODO: Handle locks in sessions as flags.
    controlLocked bool
    // host having control when IMPLICIT_CONTROL is active
    hasControl    string
}

Later check hasControl to ignore events coming from user not having control and switch control on click event.

func (manager *WebRTCManager) handle(id string, msg webrtc.DataChannelMessage) error {
    isHost := manager.sessions.IsHost(id)
    if (!manager.config.ImplicitControl && !isHost) || (manager.config.ImplicitControl && !manager.sessions.CanControl(id)) {
        return nil
    }
        ... 
        buffer = bytes.NewBuffer(msg.Data)

    hasControl := manager.sessions.IsHasControl(id)
    if (!hasControl && handler.Event != OP_KEY_DOWN) {
        // Ignore event as user don't have control and is not a "possible" click
        return nil
    }
        ....
        if payload.Key < 8 {
            err := manager.desktop.ButtonDown(uint32(payload.Key))
            if err != nil {
                manager.logger.Warn().Err(err).Msg("button down failed")
                return nil
            }

            manager.logger.Debug().Msgf("button down %d", payload.Key)

            if (!isHost) {
                // Switch control
                manager.SetHasControl(id)
            }
        } else if (hasControl) {
            err := manager.desktop.KeyDown(uint32(payload.Key))
         ....

I think in this way you must "click" to take control. Comments?