MouseLeave event triggered when clicking on the canvas embedded in the VncScreen component

[manekenpix]

On Chrome-based browsers, clicking on the embedded canvas that shows the stream (inside of this div) triggers a mouseleave event, which is handled by handleMouseLeave, and subsequently the div loses the focus.

This makes it impossible to interact with the stream after clicking on it until a mouseenter event is retriggered (by for example moving the mouse again).

[roerohan]

@manekenpix thanks for raising the issue, I think I understand what you're trying to explain, but for the sake of clarity, could you explain it in some more detail?

Also, would you like to suggest any possible fixes for the same?

P.S. for the sake of testing, you can use this site: https://roerohan.github.io/react-vnc/

[manekenpix]

@roerohan I can provide an example using your demo website:

• I connect to a VM running W10

• I click on the div where the stream is displayed so I can enter my pin (wthout moving my mouse after clicking)

This triggers a mouseleave event, so the div loses focus and when I type my pin no keystrokes are sent (and nothing is displayed in the textbox). Moving my mouse triggers a mouseenter event (bringing the focus the the main div) and now I can enter my pin.

Another way to test this is by logging something when clicking on the main div so you'll see when a mouseleave event is triggered.

Let me know if I should provide more examples or more details about this issue. Thanks!

[roerohan]

@manekenpix thanks for the description. I understood what issue you're facing now. I tried to reproduce it on the hosted demo site but wasn't able to. I'll try it out locally and push a fix if I'm able to reproduce it within a few days.

Thanks for reporting the issue 😄

[manekenpix]

@roerohan did you use a Chrome-based browser? From what I've seen, it doesn't happen when using Firefox.

[roerohan]

Hi, I tried it on Chrome (Windows 10), and Chromium (Arch Linux and Manjaro). I thought I reproduced it once or twice, but it seems to work for me. Maybe I'm doing something wrong. I'll get myself some more time later this week (or maybe next week) and look into it properly.

[daanlenaerts]

I'm actually having the same issue. It can indeed easily be debugged by adding an onmouseleave event to the canvas. Every time you click on it, the event is triggered. Did you find a solution to this @manekenpix ?

It is, however, important to keep the mouse still right after clicking, otherwise you won't notice the effect.

[daanlenaerts]

I devised a temporary fix that seems to work. By listening to canvas mouseout events and putting the focus back on the canvas, this issue doesn't seem to occur anymore.

document.addEventListener('mouseout', function (e) {
  if ((e.target as any).tagName === "CANVAS") {
      (e.target as any).focus();
  }
});

[manekenpix]

@daanlenaerts I didn't have time to try to find a solution back then. I think your fix should do it, did you try to submit a PR so @roerohan can take a look at it?

[daanlenaerts]

@manekenpix I didn't submit a PR as this quick fix of mine doesn't really seem like an appropriate solution to me. This is implemented outside of the react-vnc package and would likely cause issues for certain applications.

[roerohan]

@daanlenaerts could you explain what you've tried to do here? When the mouse moves out of the canvas, why do we try to focus on the canvas?

[daanlenaerts]

@roerohan This is more of a quick fix than anything else, but it seems to do the job.

As discussed in this issue, a mouseout event is triggered every time the user clicks on the canvas, after which no keystrokes are registered anymore as the canvas is not in focus. When the user moves their mouse again (over the canvas), it is brought back to focus and keystrokes are picked up again. This results in a very bad user experience.

The underlying issue, however, is that a mouseout event is triggered upon clicking the canvas, which shouldn't happen of course. But as this is something I have no control over because it happens within the react-vnc package, I am just bringing the canvas back to focus when this mouseout event is raised. A side effect of this is obviously that focus is brought back to the canvas when the mouse actually leaves the canvas, but this doesn't seem to be causing any issues.

Hopefully this clears things up, if not feel free to ask.

[richardstephens]

I spent some time debugging this - it looks like what is happening is that, on click, noVNC sets up a prxoy element covering the entire page. This element is transparent and only briefly in the page, but this is what is triggering the mouseLeave event.

I narrowed it down to commenting out line 101 (proxyElem.style.display = "";) in src/noVNC/core/util/events.js stops the spurious events from being fired, but this also makes the proxy element pointless. It's not entirely clear to me why any of the proxyElement logic in noVNC is necessary (perhaps a holdover from older, buggier, browsers?) as we're already in a context where we've recieved the event.

For now I'm going to work around this problem by patching handleMouseLeave in react-vnc to drop the spurious events, however I'm not sure if this fix should be upstreamed because in my testing it would also drop legitimate events, even in Firefox. Hopefully I'll get some more time to dig into it in the future.

My workaround is here