Implement modeless scrolling / clicking

[GoogleCodeExporter]

Having to manually switch between scroll-mode and click-mode is not very 
iPhone-like. What we 
want is a seamless experience where we can click to control the server and 
drag/pinch to scroll. 
Here's a suggestion how to do it, feedback is very welcome.

- When finger is pressed down, don't send a mouse down event for the first x 
milliseconds 
(about 100ms, exact number needs to be found by using it). If user moves his 
finger during that 
time, proceed with scroll handling until finger is lifted again. If user adds 
more fingers, proceed 
with zooming/scrolling until all fingers are lifted again.

- If finger is pressed down for more than x milliseconds, send a mouse down 
event. (Note that 
100% of all well written apps and 99.9% of the bad written apps all don't react 
to mouse down 
events - they wait for the mouse up event.) Now wait and watch what the finger 
does. If it stays 
within area y (about 20x20 pixels, exact number needs to be found by using it), 
don't scroll, 
don't send any info back (trembling finger correction). If finger is lifted 
within that area, send 
back a mouse up event. If finger is moved further away than the area, we need 
to send a "mouse 
up cancel" event and then proceed with scrolling.

- A "mouse up cancel" event can actually be sent even though there is no such 
thing in the VNC 
specs: move the mouse to the most remote screen corner, then immediately send a 
mouse up 
event.

This idea doesn't include support for dragging. This could be added by checking 
for a click + 
hold gesture.

Comments?

Original issue reported on code.google.com by benjamin...@gmail.com on 18 Dec 2007 at 10:23

[GoogleCodeExporter]

Yeah.. Chris wanted to have the effect of using a TABLET PC.
So a finger on the screen would control the mouse and be a down event at that 
location.
This would enable draging selection in text or draging items across the screen 
for
move or copy.

Chris and I have been going back and forth on the best way to implement this and
maintain the functionality of both.. It doesn't seem like we've been able to 
pull
that off.  We'll have to try a PLAN B.

thanks for your precise comments.

Original comment by glennkre...@gmail.com on 25 Dec 2007 at 5:50

[GoogleCodeExporter]

Yes, I very much appreciate your thorough comments!

Like Glenn said, my goal has been to have a 1-to-1 correspondence between local 
and remote action. This is 
really how the RFB protocol used by VNC is intended to work, which is why it 
sends separate up/down clicks and 
keys instead of a "double-click here", "drag from here to here" type command. 
But I understand where you're 
coming from. VNC is also intended to have a full set of user input devices on 
the local side, so it obviously breaks 
down a bit when you don't.

My plan has been to get version 1.0 released with the current mechanism and 
then go into improvement mode. 
We'll add a preference to switch between two input modes; the current and this 
new one. Before that can happen, 
though, some of the event management code needs to be heavily refactored from 
the original Cotvnc structure. 
Otherwise supporting two (or more) input modes will be a nightmare in the 
source code.

Btw, vnsea does already have a delay, currentl 285 ms, on mouse down before it 
sends the down event. This 
delay is used to wait for the user to put a second finger on the screen to 
initiate scrolling/zooming. If the finger 
moves during the delay, the mouse down is sent and remote action is locked in 
until the mouse up. Only if the 
second finger appears during the delay does scrolling/zooming start.

Original comment by crf...@gmail.com on 30 Dec 2007 at 10:39

• Changed state: Accepted

[GoogleCodeExporter]

Hi, thanks for your replies (and sorry for answering back so late). Interesting 
to hear about that 285ms delay, 
very thoughtful of you to add this!

My idea comes from the big size difference of iPhone and most target systems. 
With that difference, you're 
constantly zooming, scrolling and clicking - and being able to do this without 
constantly switching modes 
would be a big Plus. I understand that my suggestion doesn't follow the RFB 
protocoll closely. I'm more of a 
usability guy, obviously :-)

I thought about my suggestion a bit more and found a way to make it simpler to 
implement, with no need to 
cancel a mousedown event on the target system:
- When finger is pressed down, do nothing. Check for movement: if more than 20 
pixels -> scroll. If a second 
finger appears: zoom
- When finger is released, and scrolling or zooming has been started: stop 
them, do nothing else
- When finger is released, and scrolling or zooming has not been started yet: 
send a mouse down event 
followed by a mouse up event.

Result: nice iPhone-like experience. Downside: no mousedown-feedback from the 
target system shown to the 
user. Upside: clean concept, easier to implement.

Comments?

Original comment by benjamin...@gmail.com on 3 Jan 2008 at 9:46