mousePressed() not firing on mobile on touch start

eyaler commented 2 months ago

Most appropriate sub-area of p5.js?

[ ] Accessibility
[ ] Color
[ ] Core/Environment/Rendering
[ ] Data
[ ] DOM
[X] Events
[ ] Image
[ ] IO
[ ] Math
[ ] Typography
[ ] Utilities
[ ] WebGL
[ ] Build process
[ ] Unit testing
[ ] Internationalization
[ ] Friendly errors
[ ] Other (specify if possible)

p5.js version

1.10.0 and 1.9.4

Web browser and version

chrome 128 and firefox 129

Operating system

android 12

Steps to reproduce this

On android the mousePressed() function does not trigger when the touch starts.

If I do a short tap - mousePressed() triggers on release

If I do a long tap or tap-and-drag - mousePressed() never triggers

I expect mousePressed() to be triggered in the same way touchStarted() would trigger (to be clear I don't have a touchStarted handler defined)

This prevents me from using mousePressed() to capture the mouse/touch location on pointer down.

Steps:

example 1 (using official example):

on android, go to https://p5js.org/reference/p5/mousePressed/
observe that the second example does not work

example 2 (custom code to isolate the issue):

on android, go to https://editor.p5js.org/eyaler/sketches/ZRS5A3rHx
run and tap the screen and observe the mousePressed() only fires on release
long-tap, and observe the event never fires
do a drag gesture and observe the event never fires

Snippet:


let value = 0;

function setup() {
  createCanvas(100, 100);
}

function draw() {
  background(200);

  // Style the square.
  fill(value);

  // Draw the square.
  square(25, 25, 50);
}

// Toggle colors with each touch.
function mousePressed() {
  if (value === 0) {
    value = 255;
  } else {
    value = 0;
  }
}

sikaili commented 2 months ago

Hi @eyaler,

Thanks for reporting this!

It looks like you're running into a common issue with mousePressed() on mobile devices like Android. Here's what’s happening:

Short Tap Behavior: On mobile, mousePressed() is designed to behave similarly to a mouse click, so it fires on release, which is why you're seeing the event trigger only after a short tap.
Long Tap and Drag: Unfortunately, mousePressed() doesn’t handle long taps or drag gestures well on mobile devices. It’s because mobile devices treat these as touch gestures, and mousePressed() is primarily designed for mouse interactions. That's why touchStarted() or touchEnded() are more reliable for capturing touch events on mobile.

Workaround:

You can use the touchStarted() function, which is made for mobile interactions and behaves as expected when the screen is tapped (or long-tapped). Here’s how you could modify your code to handle touch events:

let value = 0;

function setup() {
  createCanvas(100, 100);
}

function draw() {
  background(200);
  fill(value);
  square(25, 25, 50);
}

// Trigger when touch starts
function touchStarted() {
  toggleColor();
}

// Trigger when mouse is pressed (for desktop)
function mousePressed() {
  toggleColor();
}

function toggleColor() {
  if (value === 0) {
    value = 255;
  } else {
    value = 0;
  }
}

Why this happens:

mousePressed() was originally designed for desktop interactions with a physical mouse.
On mobile devices, touch events like touchStarted() and touchEnded() provide better control over tap and touch gestures.

By using touchStarted(), you'll get consistent behavior across both Android devices and desktops.

Let me know if this works for your case!

eyaler commented 2 months ago

Thanks. I do find this quote from the docs to be confusing:

On touchscreen devices, mousePressed() will run when a user’s touch starts if touchStarted() isn’t declared.

Some questions:

Is the pattern below recommended?
If so, why not add this as the standard behavior? (when mouse functions are defined and touch are not)
Alternatively, could we consider adding new unified pointer functions as in JS?

The bottom line is that I expect my pointer logic to work an with either mouse, trackpad or touchscreen. Would be nice to have this out of the box or have a streamlined way to do so.

function touchStarted() {
    mousePressed()
}

function touchMoved() {
    mouseDragged()  // note: mouseDragged not mouseMoved
}

function touchEnded() {
    mouseReleased()
}

function mousePressed() {
    // actual logic
}

function mouseDragged() {
    // actual logic
}

function mouseReleased() {
    // actual logic
}

davepagurek commented 2 months ago

Also linking this to this other related issue: https://github.com/processing/p5.js/issues/7260

inaridarkfox4231 commented 2 weeks ago

p5.js doesn't seem to have any intention of fixing this bug...

davepagurek commented 2 weeks ago

I don't think that's the case, the maintainers right now are deep in the middle of some refactor for 2.0 so it might take some time before any of us get a chance, so in the mean time if anyone else wants to take a crack at it is more than welcome!

I think the first step, if possible, is to try to adhere to the documented API but with an implementation that fixes the current bugs without breaking the previously fixed ones again. this is definitely challenging and likely needs help testing on different devices, and a more systematic approach to tests so we avoid going in circles breaking prior behavior again. We haven't updated docs to reflect the current buggy situation because, if possible, this would be the route we take.

That said, we're working against the browser here, so if we try that and keep running into issues, I think a possible step 2 is to change the API and no longer try to make touches and clicks with differently than the way they work in native APIs. that's more work for users, but at least is consistent with other JS touch handling, and would be simpler to maintain.

processing / p5.js