Prevent mouse events from being captured by a live sketch when the cursor is not hovering it

[SableRaf]

Current behavior Mouse clicks register in interactive sketches even when the cursor is not over the sketch window (like clicking on links anywhere in the page). This is particularly problematic in examples like FileIO/SaveOneImage where any click triggers a file save.

Expected behavior Mouse clicks only register when the cursor is over the sketch window

Possible solution Embed the sketches into an iFrame. See @trikaphundo's first comment below

The following examples are using MousePressed to capture clicks and may need to be fixed

• Camera/Orthographic
• Input/MouseFunctions
• Structure/Loop
• Structure/Redraw
• Web/EmbeddedLinks
• Avanced Data/ArrayListClass
• Advanced Data/LoadSaveJSON
• Advanced Data/LoadSaveTable
• Cellular Automata/Wolfram
• FileIO/SaveOneImage
• GUI/Button
• Motion/MovingOnCurves
• Simulate/Flocking
• Simulate/ForcesWithVectors
• Simulate/MultipleParticleSystems

[trikaphundo]

Current behavior Mouse clicks register in interactive sketches even when the cursor is not over the sketch window (like clicking on links anywhere in the page). This is particularly problematic in examples like FileIO/SaveOneImage where any click triggers a file save.

Actually, I realized this when I was writing some examples in P5js (PR #309). Moreover, sketches are aware and receive any input event—either from keyboard or mouse— that occurs in the webpage.

This is expected, since the canvas is wrapped in a div element and directly inserted in the html element of the page. This can be checked with the browser inspector.

Expected behavior Mouse clicks only register when the cursor is over the sketch window

A possible solution (I am still investigating and experimenting on this) would be to wrap the canvas in an inner html document, that is, wrap it in an iframe element and insert that in the html element of the example page. This is what the P5js web editor does.

To make myself clearer, go over to the P5js web editor, paste the following code, play around and inspect the page's code. Note how the sketch stops updating the line when the mouse leaves the iframe element it is wrapped in.

function setup() {
  createCanvas(400, 400);
}

function draw() {
  background(220);

  line(0, 0, mouseX, mouseY);
}

Summing up, this is a problem that has very little to do with sketches; but with the html structure that holds the canvas the sketches draw on.

[SableRaf]

Thanks @trikaphundo! Shall I assign this one to you? Otherwise I can tag this as "help wanted"

[trikaphundo]

Thanks @trikaphundo! Shall I assign this one to you? Otherwise I can tag this as "help wanted"

I am writing some tests and tinkering with them, but for now I am making no promises ^^'.

Althought I have knowledge of web development technologies, never actually used an iframe. My plan is to get a small proof-of-concept repo (if I succeed) that everyone can copy and test locally, and then decide whether (and how) to take that into account for the actual processing's website.

So for now leave this unassigned, or at least do not assign it to me ^^'. I will do my best

[trikaphundo]

Hello, I have been reading the code of the P5js library that manages input events and, in particular, updating the mouse position. According to the documentation for the mouseX property

The system variable mouseX always contains the current horizontal position of the mouse, relative to (0, 0) of the canvas. The value at the top-left corner is (0, 0)

What the code actually does is computing the difference between the mouse position in the window (clientX and clientY properties) and the top left corner of the bounding rectangle of the canvas, which is a point in the page of the document.

Therefore the question is: where is our p5 sketch getting the events from? That is, who are the event handlers being registered to?

The 'problem' with events

The events P5js offers are binded to the window object of the html document (see the code). Note that this applies to both, mouse events and keyboard events. Therefore, it is perfectly normal that sketches respond to mouse and keyboard events occuring in the window, irrespective of whether the canvas has the focus or the mouse is over it.

It is interesting to note, though, that p5 objects register their event handlers to the window of the document they belong to, instead of the canvas' container div, because registering them to the div makes keyboard events not work. This is stated in a comment in the code linked to by the first link of the previous paragraph.

// Bind events to window (not using container div bc key events don't work)

This suggest a solution: give the sketch its own window.

Solutions

I see two possible solutions:

1. Purposely ignore events in every single sketch we write, when the mouse is not over the canvas, or the canvas has not the focus. Really ugly solution but does not require modifying the website, just the sketches with boilerplate code.
2. Load P5js library and the sketch in a separate window embedded in the document, so that the p5 object registers the event handlers to that inner embedded window. This can be done with the html element iframe.

I am dismissing the former and explaining how to carry out the latter. For a working example of this latter solution, see further below.

The changes to the existing code of Processing's website will be the following (I have no idea about React, so more changes may be in order)

• Html code of the examples template page
  • Keep the div with id example-cover that currently holds the canvas element. If no sketch is available, load the image. This is the current behaviour.
  • If a sketch is available, insert an iframe (specifying its dimensions in absolute units, not relative or unspecified) as a child of that div. This iframe loads an html document that just loads the P5js library and the sketch, nothing more than that.
• CSS style must be applied to the iframe element: by default it is an inline element, it should be changed to block: display: block. Also remove its borders with border: none.
• Sketches no longer set explicit dimensions to the canvas, but rather fill the window they belong to: must substitute the call createCanvas(640, 360) for createCanvas(windowWidth, windowHeight), so that the canvas perfectly fits its window (the iframe). The exact dimensions (640x360px) must be given to the iframe element, either through CSS or by setting its width and height attributes. This is because iframe does not behave as other block elements, which can adapt its dimensions to their content.
• CSS style must be applied to the html document the iframe will display, namely: body, main and canvas elements, in order to prevent the appearance of scroll bars in the iframe. By default the canvas element is an inline element, it should be displayed as a block: display: block. For the body and main elements set border: none; padding: 0; margin: 0. This will make the canvas completely fill the iframe.

There is no actual need to store in the server a new html document with just a pair of script tags, one to load the p5js library, the other to load the sketch, to later on give its URL to the src attribute of the iframe element. The document is so simple it can be embedded in the iframe's srcdoc attribute, which allows feeding in html content as a string.

Thus you can create the html document as a string in your react code and inject that string in the iframe's srcdoc attribute. This document consists of

• Script tags to load the P5js library and the sketch.
• Style tag with CSS code to carry out what has been outlined in the latter point of the previous list.

Note: as the sketches are loaded in example pages by means of useEffect, the script tag for loading the sketch may be dropped? Someone with knowledge of react should work this out.

Minimal example

I have created a repository with a minimal working example. In this example I have not used the srcdoc attribute, but src. Anyway you should have no trouble using it by following through with the instructions given above.

Sandbox

In my example repository no Internet connection is required, but if you do load p5js from the Internet from within the iframe, you will encounter some problems. They have to do with security measures html enforces for embedded content; in such case you have to lift the appropiate restrictions on the iframe element via its sandbox attribute. Which restrictions to lift is up to the maintainers of the website.

---

I hope this is clear enough so that someone can turn my working example into working code for the Processing's website. There is the detail of how to load p5js and the sketch in the iframe with react; that will have to be worked out by whoever takes on this.

[SableRaf]

Thanks for the detailed investigation!

Regarding your point about createCanvas(windowWidth, windowHeight) I suppose we would also need to do the following to support the responsive layout of the website:

function windowResized(){
    resizeCanvas(windowWidth,windowHeight)
}

I think I'll implement the ugly solution for the FileIO/SaveOneImage example as a quick fix, but it's good to know that we have a path for a more robust solution.

[trikaphundo]

Thanks for the detailed investigation!

Whether Processing ends up adopting this proposal or not, I believe it is best to write down why and how it works; so that anyone can use this information in the future :).

---

Regarding your point about createCanvas(windowWidth, windowHeight) I suppose we would also need to do the following to support the responsive layout of the website:

function windowResized(){
    resizeCanvas(windowWidth,windowHeight)
}

Not unless the size of the iframe changes. The size of the iframe is fixed in absolute units (640x360px), and the canvas is instructed to fill its (document's) window (the iframe). Therefore it does not matter if the top window is resized, the canvas only cares about its window (the iframe), and as long as the iframe's size remains unchanged there is no need to use that function.

You can try this with the web browser inspector: open the inspector and manually change the dimensions width and height of the iframe element (via CSS or HTML attributes, it does not matter). You will see that, either scroll bars appear, or the canvas does not resize to fill the resized iframe.

---

I think I'll implement the ugly solution for the FileIO/SaveOneImage example as a quick fix, but it's good to know that we have a path for a more robust solution.

In issue #314 I mentioned this very issue. I did so because the version I wrote in P5js for the scale shape example, does not exhibit this behaviour of responding to events happening in the whole window, in that example, instead, it only responds to the mouse movement when it hovers over the canvas.

That is interesting because in that example, the onmousemove event handler is not registered with P5, but with Paperjs Tool object. Therefore I think there may be another way of accomplishing this with P5js sketches without having to modify the website code, but I have to look up the Paperjs code.