Incorporating pointerLock API to this library

[ffd8]

I've found the mouseX/Y pan/tilt to be rather tricky, especially when one runs into the browser window edge.. In my current usage of the library, I introduced an edge and edgeSpeed for continuing the pan/tilt in that direction if running out into window edge, but this is also frustrating since the speed doesn't match the mouseX-pmouseX relationship. I recently experienced a nice 3d environment built with three.js, which let one switch between really limited 'n00b' style controls with the arrow keys and the expected WASD + mouse, which was super smoooth and didn't have the issue with the mouseX/Y running out of the window = found it's based on the three.js pointerLock example

Started checking out the pointerLock API, to discover that p5.js already has it integrated! requestPointerLock + exitPointerLock = made an extended p5.RoverCam example with this feature:

https://editor.p5js.org/ffd8/sketches/5-4Zh2ciy (click once into the canvas to toggle it)

This brings up the question of how to best integrate this to the lib?? I find the pointerLock controls soooo much more fluid and intuitive than the current mapping of mouseX/Y movements... but of course, one loses their cursor for any potential interaction with the environment or HUD... might be weird to have no mouse pan/tilt until activating the pointerLock... and if using this mousePressed toggling, then the use of a mousePress (moving forward) within the lib should be de-activated (something that still needs optomization for mobile devices, only being able to lunge forward). What do you suggest? In my own implementation of the library, I've now added z and c to pan left and right for additional key options. One problem with using the mousePress to toggle it, is ESC is also a standards way to exit the mode.. so an eventListener would need to be activated to catch exiting via ESC to avoid the libs pointerLock boolean to appear active when it's not.

The example above handled it quite well (maybe this is default for three.js??) – when first landing in environment, the mouse is only a cursor (no pan/tilt) and A/D (left/right) act as pan, while W/S (up/down) are move forward/backward (no tilt).. if either clicking button, or canvas, or pressing hot-key, one could toggle into pointerLock mode, with existing/gamer ADWS/arrow_keys for movement only and mouse with endless pan/tilt.

Edit: only took a few more lines of code to make an example of the scenario above: https://editor.p5js.org/ffd8/sketches/sJesWkOel

[jwdunn1]

Really interesting! Thanks for your insights here. This has me investigating alternative library-design patterns to allow a developer to override the default interface... perhaps using a controller method. Controls are a user preference: from keyboard to mouse to touch to HUD. BTW, after experiencing the examples you provided, I prefer panning using left/right arrow (or AD) keys. I also like the touch interface in FPS games like Battle Bay. Other people prefer space to move up and shift to move down. Some may prefer pointerLock, others may not, thus (with a little re-engineering of the library) it should be easy to handle that at the application level.

[jwdunn1]

In this experimental sketch, I've re-engineered the library as I mentioned. Pointer lock is handled at the application level, with default behavior being overridden using the new library methods rotationController() and translationController(). These methods allow the implementer to define key/mouse/touch behaviors utilizing the primitive camera control methods. BTW, the app handles the escape condition using an event listener. Any feedback on the library architecture would be appreciated.

[ffd8]

Aha that's a good idea to let as many aspects as possible be overwritten at the application level. From the example you shared, atleast on my computer, the POV movement crawls, but I see that's just due to the sensitivity value now placed within the movements. Within the lib, I see the default sensitivity is now a fixed value.. but I'd keep it as a variable, sensitivity or similar name, so the default setup can be used while easily adjusting the pace.

Great you could implement ESC to also toggle the pointerLock mode.

Here's a sketch with my latest implementation of the lib (prior to newest dev) and app implementation (just search sketch + lib for ffd8 to see additions) – main things added are pointerlock, running, respawning, elevation, hud-info parsing, fading cursor, and a few other random bits.. but I'll try to refactor it into the newer version you posted above, since those changes make a lot of sense– keep the lib as minimal as possible, and let the application usage overwrite key aspects.

[jwdunn1]

I like the updates in your last sketch. Thanks for the feedback...good use case to leave sensitivity and speed within the lib. In this next sketch, I've made a few changes toward simplicity. Only one controller is necessary, thus the two are now combined into one. I've consolidated and renamed the primitive control methods to reflect the 6 degrees of freedom from the first person perspective (moveX=forward/back, moveY=left/right, moveZ=up/down, yaw=pan, and pitch=tilt). Eventually roll needs to be implemented for flight simulation or banked curves. I know these changes will be yet another refactor, but the changes are minimal. If they work for you, I can update the repository.

[ffd8]

Looks good – tilt/pan are much easier for a layperson (to flight terminology) to understand than pitch/yaw, but as roll is introduced, makes sense to switch. Merged controls as one function looks good.

It would be nice if some of the application code could be offloaded but customizable to the lib.. wondering if the whole pointerLock toggle code/vars could/should be part of the lib? Not sure if it's possible to attach to the mousePressed/mouseClicked function (especially if it's not yet declared within the sketch?) – but that logic of toggling between keys + pointerLock seems useful enough for every application, would be ideal to not have to set it within each app usage. That would then also involve adding some of the app code for control back (w/ if statement) back into the lib.

The added perspective modification is great and can be quite trippy! Seems like it should also be a variable in the library which a keypress (ie +/-) just change, especially since a window resize would overwrite that perspective. Same goes for near and far cutoffs.. which I'm guessing help rending speed if cutting off view of objects x distances away from POV? Here's a modified sketch from yours above – not sure about name of new vars (view object) or pov() as updating perspective.. but basic idea could be useful?

[jwdunn1]

I agree with moving the pointerLock code into the library. Working on that now here.

[jwdunn1]

Latest improvements here 1) Moved pointerLock into the library (required passing the canvas as a config parameter) 2) Made pointerLock optional (also passing in as a config param) 3) Added fov() method and mapped to +/- keys in the default controller 4) Added reset() method 5) Added shift key for speed doubling (at the app level) 6) Added player containment (at the app level)

If usePointerLock is false, then navigation is keyboard-only in the default controller.

[ffd8]

Awesome all around! Works really smooth in my quick testing.

• Is there a specific reason to have the canvas.elt in the application, rather than just grab the .elt within the lib? Seems more user-friendly to not deal with .elt (I can only think of one instance where I've had to reference it). Suggestion, Line 42 of the lib, changing to this.canvas = config.canvas.elt;, and on app level, just sending canvas: canvas within the Player object.

• Flying keys (by default in lib) seem pretty fast.. * 2 matches rest of sensitivity more than * 5 (lines 94 + 95)

• Found bug on flying.. with new lib embedding e and q as defaults, can disable app lines 177 - 180 (e for flying) – it causes the speed of gravity to act weird if trying to drift in any direction while floating down.. once disabled, speed is normal while falling in a given direction.

• Found a bug on app, (line 184), space to jump, should be this: if (this.grounded && keyIsDown(32)) { // space – using keyIsDown() otherwise, if you move forward, press space.. let go of space but keep moving forward.. endlessly hops like a bunny...

• What were your thoughts on adding the elevation variable to the lib, attached to the camera? I found it interesting to enable a mouse or giant POV (while still respecting hit dection of app). It was part of this sketch. Maybe there's a better word than elevation... cameraHeight.... cameraY? Doesn't need default keys to adjust it.. since maybe it's only set once (ie. 🤖 vacum simulation..??).

• Falling out of the maze is one of my favorites parts! Haha.. but sure, some applications may want that..

[jwdunn1]

Latest. Refactored pointerLock. Problems arose with multiple instances of RoverCam - realized it needs to be a one-time global option.

1. There is now an optional global method RoverCam.usePointerLock(renderer) to enable pointerLock for all camera instances.
2. Fixed the fly speed.
3. Not seeing any issues with gravity acting weird if trying to drift...the Q key simply adds to gravity.
4. Fixed the bunny hop bug in the maze app.
5. Elevation is an app thing: adjust the player y dimension at line 161. (for example, 0.2 is mouse height and 12 is nice for looking over the walls)
6. Added methods setState() and setActive() to support switching of instances including proper handling of perspective changes between them. Use keys 1,2,3 to switch cameras.

Not perfect yet, but it's getting there.

[jwdunn1]

Interestingly, pointer lock is not presently allowed in OpenProcessing. A sketch runs in a sandbox which does not have "allow-pointer-lock" permissions enabled. I've submitted a change request.

[jwdunn1]

Update. Refactored the entire library for p5js multi-instancing - required cleaning up stray global references. Latest maze app. Remove argument passing to RoverCam.usePointerLock(). Multi-camera test. Use keys 1,2,3 to switch cameras. Multi-instance test. Use space key to toggle between controlling canvas 1 and canvas 2. This is the only time when the instance variable needs to be passed to RoverCam.usePointerLock() I will experiment later today with moving usePointerLock() back into the class methods.

[ffd8]

Awesome it's refactored for instance-mode! Looks like a great solution and the setState is really nice for initiating the camera positions. Both new tests (multi-cam/instance) work smoothly. Here's a quick test tweaking your multi-instance for a side + top view... probably a more efficient way to do that (and only draw the actual shapes once).

Re: y dimension vs elevation, an issue is it quickly falls through the blocks somehow avoiding hit-detection.. but that's an app logic issue. Benefit of elevation was changing POV while keeping basic 1:1 block calculations. One strange thing is since this refactor, I break into and sometimes through the blocks (especially if running).. so maybe the refactor changed enough that the hit logic is now missing. (tiny detail)

Re: iframe + allow-pointer-lock permission – yeah, I ran into the same issue with P5LIVE – but it's now allowed in pending update.

[jwdunn1]

I've tweaked my multi-instance sketch to demonstrate the use of a common scene rendering function. Regarding break-into/through the blocks, I went back and tested earlier versions and found the same problem...primarily related to running. I saw this in the original version too and believe it to be an artifact of the rudimentary collision detection section of the app and the method by which the blocks are arranged with some of them pushed up or down. A smaller dimension of the player literally slips between the cracks (probably more often when the dimension is less than 0.5). Perhaps a global floor check is needed to prevent piercing, similar to the global edge checks (not with a player reset, but just a push back to allowed values). New version of the library includes developer-redefinable keymap, elevation (r/f keys in pLock mode), and moved pointerLock back into the class methods. Elevation is implemented at part of ocular offset which allows for stereo vision apps. Cool to be able to navigate around 3D objects!

[ffd8]

D'oh - haha of course, using a function with code and passing the p5 instance is clever– great technique to have shared code for both views.

New keymapping is a good idea + thanks for considering/including elevation, great what becomes possible also having a horizontal offset. This ocular effect is awesome! Definitely paves the road towards eventually enabling onDeviceMotion for usage with headsets/cardboard – will look into if/how it can be combined with funtions from p5.xr.

Re: going through walls = ok, definitely has to do with the speed.. changing run to .06 prevents it.. so yeah, just a matter of app/maze logic not the lib.

[jwdunn1]

If the commit checks out, I believe this issue can be closed.

[ffd8]

Sorry was AFK – but looks great (testing now, so far so good.. just strangely in my app usage, the sensitivity/speed are far different.. my own issue)

Nice pingponging these additions to the lib + the expanded list of examples in the readme is great!