MX Ink support - Githubissues

Utopiah commented 1 week ago

Description:

The Logitech MX Ink is out. It's basically another controller with the "shape" of a pen and different button mapping. It would be good to support it. I'll tinker with it but if others have one and want to contribute information, or directly code, this could be a starting place.

A-Frame Version: latest
Platform / Device: Quest2, Quest3, Quest3s
Reproducible Code Snippet or URL: https://logitech.github.io/mxink/GetStartedWithStylus.html

Utopiah commented 1 week ago

Note that the official documentation does include a 3D model already, albeit not glb.

Utopiah commented 1 week ago

logitech_mx_ink.glb.zip

Utopiah commented 1 week ago

Working WebXR demo on needle https://engine.needle.tools/samples/pen-and-sword/

Utopiah commented 1 week ago

Getting gamepad buttons presses on

0 as lower button on pen, values going from 0 to 1,
1 as middle button on pen, only 0 or 1, and
4 as upper button on pen, only 0 or 1.

I don't know about the pen tip though.

mrxz commented 1 week ago

The immersive-web/webxr-input-profiles repository already has a profile file for the MX Ink, which includes the gamepad mappings: logitech-mx-ink.json.

But it seems to suggest that index 0 is the upper button (towards tip), 4 the middle button and 1 the lower button. Index 5 would be the tip-force. :thinking: @Utopiah could you try the WebXR Input Profile Viewer and see if the right buttons move when you press them in VR? Wouldn't be the first time that the input profiles in the repository are wrong...

Utopiah commented 1 week ago

Thanks, unfortunately I don't see any value updating, neither in 2D in the HMD nor via remote debugging. I'm not sure how I can check from the viewer.

I don't see any visual update on the model when I force it. When I use the controller model some buttons do move, but not the same way as in AFrame (nor my console AFAICT).

mrxz commented 1 week ago

Thanks, unfortunately I don't see any value updating, neither in 2D in the HMD nor via remote debugging. I'm not sure how I can check from the viewer.

The viewer is a bit confusing in that regard. The 2D input elements on the page never "output" anything, they are strictly input to manually control the models. But if you enter the immersive session it should load the relevant model(s) for the input sources you have. Then when you press any of the buttons, they should animate the corresponding component on the model itself. The effect might be subtle so you may need to hold the pen close to your eyes.

Based on the input profile index 4 would be the middle button, however, you noted that pressing the upper button corresponds to index 4. So inside the viewer, if you physically press the upper button, does the upper button on the 3D model move or does the middle button move (or even something else).

And if the values are truly only 0 and 1, then the buttons wouldn't really animate but 'jump' between those two extremes when pressing.

Utopiah commented 1 week ago

~~Thanks for the clarification. Unfortunately despite few refreshes it does not ... well let me reboot the device, just in case.~~

Ah! Somehow WebXR sessions in the browser can be unpredictable and I found out that relatively often (like I lose hands...) rebooting helps. So now it does work! The mapping seems fine and correct, i.e. the right model loads and the button presses match the model.

Utopiah commented 1 week ago

BTW I'm also getting value for the 2nd (large) button now, not "just" 0 or 1.

Utopiah commented 1 week ago

I can't get the pen tip value though using console.log( AFRAME.scenes[0].xrSession.inputSources[0].gamepad.buttons.map( b => b.value ).join(',') ) whereas the other 3 buttons do work.

dmarcos commented 1 week ago

Thanks. When ready feel free to submit a PR with a logitech-mx-ink-controls component

dmarcos commented 1 week ago

I don't have a device so can't test

Utopiah commented 1 week ago

So, the plan then would be :

clone src/components/oculus-touch-controls.js but limited to a single controller, the MX Ink one, probably based on oculus-touch-v3
use https://github.com/immersive-web/webxr-input-profiles/blob/main/packages/registry/profiles/logitech/logitech-mx-ink.json
use the provided glb, insuring that buttons can be activated (seems to be the case in the WebXR Input Profile Viewer but might be another model)
provide a test URL

hybridherbst commented 1 week ago

In case it helps, the buttons are at index [4] and [5] (4 is stylus-touch and 5 is stylus-tip). First entry in the profile is logitech-mx-ink. Make sure you're on the latest OS version – these indices have been shuffled around multiple times in the last OS releases...

Utopiah commented 1 week ago

To try live https://aframe-mx-ink.glitch.me without remapping, which AFAICT doesn't take stylus-tip into account.

dmarcos commented 3 days ago

@hybridherbst It looks the buttons indices exposed by the browser don't match the profile?

I would expect

"buttons": [
    "xr-standard-trigger",
    "xr-standard-squeeze",
    null,
    null,
    "touch-pad",
    "tip-force",
    "docked"
],

hybridherbst commented 3 days ago

Hm, for me that profile matches what I see on device – button 4 is touch-pad and button 5 is tip-force. Neither of them is trigger/squeeze. Which OS version are you on?

dmarcos commented 3 days ago

@hybridherbst The pen has three buttons + tip (not counting the meta button). Are they not all exposed?

I manage to render the model but it doesn't align. Are you using gripSpace?

Thanks

dmarcos commented 3 days ago

Never mind alignment. I was not using gripSpace properly

aframevr / aframe

MX Ink support #5583