Open SableRaf opened 2 years ago
Current behavior Mouse clicks register in interactive sketches even when the cursor is not over the sketch window (like clicking on links anywhere in the page). This is particularly problematic in examples like FileIO/SaveOneImage where any click triggers a file save.
Actually, I realized this when I was writing some examples in P5js (PR #309). Moreover, sketches are aware and receive any input event—either from keyboard or mouse— that occurs in the webpage.
This is expected, since the canvas is wrapped in a div element and directly inserted in the html element of the page. This can be checked with the browser inspector.
Expected behavior Mouse clicks only register when the cursor is over the sketch window
A possible solution (I am still investigating and experimenting on this) would be to wrap the canvas in an inner html document, that is, wrap it in an iframe element and insert that in the html element of the example page. This is what the P5js web editor does.
To make myself clearer, go over to the P5js web editor, paste the following code, play around and inspect the page's code. Note how the sketch stops updating the line when the mouse leaves the iframe element it is wrapped in.
function setup() {
createCanvas(400, 400);
}
function draw() {
background(220);
line(0, 0, mouseX, mouseY);
}
Summing up, this is a problem that has very little to do with sketches; but with the html structure that holds the canvas the sketches draw on.
Thanks @trikaphundo! Shall I assign this one to you? Otherwise I can tag this as "help wanted"
Thanks @trikaphundo! Shall I assign this one to you? Otherwise I can tag this as "help wanted"
I am writing some tests and tinkering with them, but for now I am making no promises ^^'.
Althought I have knowledge of web development technologies, never actually used an iframe. My plan is to get a small proof-of-concept repo (if I succeed) that everyone can copy and test locally, and then decide whether (and how) to take that into account for the actual processing's website.
So for now leave this unassigned, or at least do not assign it to me ^^'. I will do my best
Hello, I have been reading the code of the P5js library that manages input events and, in particular, updating the mouse position. According to the documentation for the mouseX
property
The system variable mouseX always contains the current horizontal position of the mouse, relative to (0, 0) of the canvas. The value at the top-left corner is (0, 0)
What the code actually does is computing the difference between the mouse position in the window (clientX and clientY properties) and the top left corner of the bounding rectangle of the canvas, which is a point in the page of the document.
Therefore the question is: where is our p5 sketch getting the events from? That is, who are the event handlers being registered to?
The events P5js offers are binded to the window object of the html document (see the code). Note that this applies to both, mouse events and keyboard events. Therefore, it is perfectly normal that sketches respond to mouse and keyboard events occuring in the window, irrespective of whether the canvas has the focus or the mouse is over it.
It is interesting to note, though, that p5 objects register their event handlers to the window of the document they belong to, instead of the canvas' container div, because registering them to the div makes keyboard events not work. This is stated in a comment in the code linked to by the first link of the previous paragraph.
// Bind events to window (not using container div bc key events don't work)
This suggest a solution: give the sketch its own window.
I see two possible solutions:
I am dismissing the former and explaining how to carry out the latter. For a working example of this latter solution, see further below.
The changes to the existing code of Processing's website will be the following (I have no idea about React, so more changes may be in order)
example-cover
that currently holds the canvas element. If no sketch is available, load the image. This is the current behaviour.display: block
. Also remove its borders with border: none
.createCanvas(640, 360)
for createCanvas(windowWidth, windowHeight)
, so that the canvas perfectly fits its window (the iframe). The exact dimensions (640x360px) must be given to the iframe element, either through CSS or by setting its width and height attributes. This is because iframe does not behave as other block elements, which can adapt its dimensions to their content.display: block
. For the body
and main
elements set border: none; padding: 0; margin: 0
. This will make the canvas completely fill the iframe.There is no actual need to store in the server a new html document with just a pair of script
tags, one to load the p5js library, the other to load the sketch, to later on give its URL to the src
attribute of the iframe
element. The document is so simple it can be embedded in the iframe's srcdoc
attribute, which allows feeding in html content as a string.
Thus you can create the html document as a string in your react code and inject that string in the iframe's srcdoc
attribute. This document consists of
Note: as the sketches are loaded in example pages by means of useEffect
, the script tag for loading the sketch may be dropped? Someone with knowledge of react should work this out.
I have created a repository with a minimal working example. In this example I have not used the srcdoc
attribute, but src
. Anyway you should have no trouble using it by following through with the instructions given above.
In my example repository no Internet connection is required, but if you do load p5js from the Internet from within the iframe, you will encounter some problems. They have to do with security measures html enforces for embedded content; in such case you have to lift the appropiate restrictions on the iframe element via its sandbox
attribute. Which restrictions to lift is up to the maintainers of the website.
I hope this is clear enough so that someone can turn my working example into working code for the Processing's website. There is the detail of how to load p5js and the sketch in the iframe with react; that will have to be worked out by whoever takes on this.
Thanks for the detailed investigation!
Regarding your point about createCanvas(windowWidth, windowHeight)
I suppose we would also need to do the following to support the responsive layout of the website:
function windowResized(){
resizeCanvas(windowWidth,windowHeight)
}
I think I'll implement the ugly solution for the FileIO/SaveOneImage
example as a quick fix, but it's good to know that we have a path for a more robust solution.
Thanks for the detailed investigation!
Whether Processing ends up adopting this proposal or not, I believe it is best to write down why and how it works; so that anyone can use this information in the future :).
Regarding your point about
createCanvas(windowWidth, windowHeight)
I suppose we would also need to do the following to support the responsive layout of the website:function windowResized(){ resizeCanvas(windowWidth,windowHeight) }
Not unless the size of the iframe changes. The size of the iframe is fixed in absolute units (640x360px), and the canvas is instructed to fill its (document's) window (the iframe). Therefore it does not matter if the top window is resized, the canvas only cares about its window (the iframe), and as long as the iframe's size remains unchanged there is no need to use that function.
You can try this with the web browser inspector: open the inspector and manually change the dimensions width
and height
of the iframe element (via CSS or HTML attributes, it does not matter). You will see that, either scroll bars appear, or the canvas does not resize to fill the resized iframe.
I think I'll implement the ugly solution for the
FileIO/SaveOneImage
example as a quick fix, but it's good to know that we have a path for a more robust solution.
In issue #314 I mentioned this very issue. I did so because the version I wrote in P5js for the scale shape
example, does not exhibit this behaviour of responding to events happening in the whole window, in that example, instead, it only responds to the mouse movement when it hovers over the canvas.
That is interesting because in that example, the onmousemove
event handler is not registered with P5, but with Paperjs Tool object. Therefore I think there may be another way of accomplishing this with P5js sketches without having to modify the website code, but I have to look up the Paperjs code.
Current behavior Mouse clicks register in interactive sketches even when the cursor is not over the sketch window (like clicking on links anywhere in the page). This is particularly problematic in examples like
FileIO/SaveOneImage
where any click triggers a file save.Expected behavior Mouse clicks only register when the cursor is over the sketch window
Possible solution Embed the sketches into an iFrame. See @trikaphundo's first comment below
The following examples are using MousePressed to capture clicks and may need to be fixedCamera/OrthographicInput/MouseFunctionsStructure/LoopStructure/RedrawWeb/EmbeddedLinksAvanced Data/ArrayListClassAdvanced Data/LoadSaveJSONAdvanced Data/LoadSaveTableCellular Automata/WolframFileIO/SaveOneImageGUI/ButtonMotion/MovingOnCurvesSimulate/FlockingSimulate/ForcesWithVectorsSimulate/MultipleParticleSystems