Cannot get a pen up event in iPad

[haviital]

It is possible to detect only pen down event (mouse key down). When i touch the screen "mouse key down" is true and then immediately false, even if I keep finger down. So there is no way to get pen up event, which is quite essential.

But Snap is however a great tool and can become the first really cross-device scratch-like framework because of javascript! Just keep doing the good work and improve the iPad functionality too.

[jmoenig]

Thanks! I cannot reproduce this. On my iPad Snap's mouseDown? reporter shows the current state of whether a finger touches the glass or not.

[haviital]

Really? A simple example is below. I have iOS 8.1 in my iPad3 3rd gen. I have tested it both on Safari and Chrome browser.

[jmoenig]

yes, I was testing the exact same script on my iPad, same iOS version. Whenever I put the finger on the glass the sprite says "true", when I lift it up again it says "false".

[jmoenig]

Here's what happens on my iPad:

[haviital]

Ok, I cannot deny that :-) I have to retest this at home with iPad today.

[jmoenig]

I have to say that the reaction is kinda slow, though... there is a distinct pause between the finger actually touching the glass and the sprite saying so

[haviital]

I double checked the problem in my iPad. If I keep the finger down for few seconds the message changes to "false", without lifting the finger. It does not happen every time. Sometimes it works as expected, but over 50% of time it works incorrectly.

The fps is also really slow in general. Something like 1 fps! It can be related to touch problems. Perhaps touch events are not handled fast enough and they return error(?).

[haviital]

I noticed that using "Say" command in Forever loop makes Snap very slow (about 1 fps) on iPad. Changing sprite costume based on mouseDown state is much cheaper.

However the original problem stays. I even took the Snap source and modified the SensorMorph example in morphic.js and made the html-test page. You can test it here: http://koti.mbnet.fi/~haviital/snap/hannu_test1.html

How to reproduce the problem: 1) Touch inside the window for about 1 sec and lift the finger. 2) Repeat about 10 times

I always get several times, where the morph stays green even after you have lifted the finger. There really seems to be a problem in the morphic.js with iPad touches.

The two changed files are here: https://dl.dropboxusercontent.com/u/54299786/ipad_test.zip

Br, Hannu

[bromagosa]

This may be related to the loop interval... same thing happened to me on a Beaglebone, and changing the interval fixed it.

I think a strategy to decide what interval goes in each machine automatically would solve many of these issues at once. Should I try to implement this?

[jmoenig]

it could be something else, entirely. @haviital observed that it's specific to the SAY command (and, I'm assuming, also to the THINK one), when used inside a FOREVER construct. Now, what this does, is constantly scanning the receiver's owner chain to find the stage, scanning for an existing talk bubble, destroying that talk bubble, creating a new one, and - here's the costly part - positioning the new talk bubble. It's the positioning that is inefficient, because it uses collision detection to come "close" to the - possibly irregularly shaped - sprite (it doesn't just align itself along the sprite's bounding box). This in itself takes some cycles, and scanning the pixels and doing the masking for collision detection is likely to be more inefficient on a mobile device anyway, even more so, if that's constantly going on.

I think the best solution to this would be to modify the SpriteMorph >> bubble() method to find out whether the value that's to be displayed in the speech bubble differs from the one that might already be shown, and only to update the bubble if it has changed.

What do you think?

[brianharvey]

That's a kludge -- you could have a FOREVER that counts down a variable and SAYs the value each time through. The values will be different, so you're back to doing it the slow way.

[jmoenig]

I don't think you understand, Brian. If you SAY a countdown value FOREVER it doesn't really matter whether it's a little slower, because it only flashes up once per display cycle anyway. If you do a count down, you'd probably want to use a SAY FOR (0.5) SECS block anyway so you can at least see what's being said. It's the continuous SAYing of same values while wanting to become notified of a change that's problematic on an iPad. In this particular case @haviital is FOREVER saying the current value of MOUSE DOWN, and that doesn't change quickly enough when he puts down or lifts a finger. So his first impression was that the "pen up event" was either not working at all on an iPad (hence the title of this issue) or processes very slowly. He then found out that the event is, in fact, instantaneous in Morphic, but the SAY block is delaying it from being immediately shown.

[brianharvey]

No, I get that, but wouldn't the problem be just as bad if the script were FOREVER IF SAY (JOIN WORDS (MOUSE X) (MOUSE Y)) or something like that? It's not the sameness of the speech that's the problem, but rather the repeatedness. No?

[jmoenig]

sure, the repetition is the problem. But how would we get a quicker reaction of a change of values in a FOREVER loop? The SAY command is blocking the execution of the script until a new speech bubble is created and positioned, only then can it repeat and - because this is applicative order - again test for the mouse button state. My point is that the test itself happens fast, and you're right that displaying the new result takes a while, but it need only take half the time it takes now, because currently the continuous SAY is slowing down the whole scheduler. So, before Snap even gets to test it is still entangled in - again - displaying an unchanged value. The user doesn't even see the continuous refresh, to them it looks as though Snap isn't doing anything, while, in fact, it's working harder than ever.

[jmoenig]

We could also try to get around using collision detection to position the speech bubble. Any good ideas how to accomplish that?

[haviital]

Hi all, even with the quick method, like changing the alpha of the morph when touched, it is still behaving incorrectly in iPad. It seems not always to register pen up event, as demonstrated with my test html file above.

[haviital]

Hi, I finally find the quilty code :-) If I remove mouse right button click simulation in morphic.js, touch down & up functionality works beautifully in my iPad! I commented the code out as follows:

HandMorph.prototype.processTouchStart = function (event) { var myself = this; MorphicPreferences.isTouchDevice = true; clearInterval(this.touchHoldTimeout); if (event.touches.length === 1) { //this.touchHoldTimeout = setInterval( // simulate mouseRightClick //function () { // myself.processMouseDown({button: 2}); // myself.processMouseUp({button: 2}); // event.preventDefault(); // clearInterval(myself.touchHoldTimeout); //}, //400 //); this.processMouseMove(event.touches[0]); // update my position this.processMouseDown({button: 0}); event.preventDefault(); } };

I suppose what happens is that the timed right click event happens at the same time (after 400 ms) as real left mouse button up (i.e. touch end). This somehow messes up the state so that left mouse button up is not registered at all.

[haviital]

Mouse right button click simulation actually always sets "this.mouseButton = 'right'" in the processMouseDown() function. So always when the 400 ms interval event is launched. But if you happen to move the finger a bit before the time is up, the interval event is cleared and touch up works! That made the bug appear so randomly.

Anyway, what is needed is a new MouseClick()-function, which do not mess with this.mouseButton variable content, like calling processMouseDown & processMouseUp in series.