Open yuripourre opened 2 years ago
@yuripourre hmm, i think locally i have updated version of latest WebXR specs ... i have to check it and push.
Very impressive, can you share a demo ? Maybe you can show your work to the community ? Each Thuesday we do a publics calls on twitch so feel free to join (check gitter gwtproject or vertispan/j2cl channels)
libgdx is based on gwt2 ? right ? i totally moved to j2cl and i do new version of three4g with focus on j2cl as well
@treblereel
@yuripourre hmm, i think locally i have updated version of latest WebXR specs ... i have to check it and push.
I would really really appreciate!
Very impressive, can you share a demo ? Maybe you can show your work to the community ? Each Thuesday we do a publics calls on twitch so feel free to join (check gitter gwtproject or vertispan/j2cl channels)
Yeah, I can share it there. I tried gitter but is so confusing :)
libgdx is based on gwt2 ? right ? i totally moved to j2cl and i do new version of three4g with focus on j2cl as well
Yes, I had to downgrade a few classes to gwt2 but I am using a also using a mix of elemental2 and gwt classes. Took me a few months but surprisingly it works.
@yuripourre are you talking about this apis ? https://www.w3.org/TR/webxr-hand-input-1/
@treblereel yes, precisely.
Please tell me if there is something I can do to help.
I had trouble defining this sequence of XRJointSpaces, I tried JsArray and XRJointSpaces[] (and a lot of combinations) but no matter what I tried they were always null. The hard part is creating a code that tries to populate them and run on a device with hand-tracking capabilities.
Because GWT obfuscates the code is really hard to understand what is wrong, too.
@yuripourre are you sure it supported ? maybe it's too new ? what is your current setup ?
@treblereel yes it is new but it is supported by Oculus Quest 2 (the device I am using for the experiments). This example works without issues: https://stewartsmith.io/handy/
@yuripourre ok, looks like i am good to go!
ps: the demo is using three.js, so ... how do you running your demo it at your Oculus Quest ?
@treblereel I wish it was simpler. Right now, I have a libGDX example project with a modified version of your WebXR library that I can compile and serve locally (using my notebook). Then I use ngrok to enable https and redirect to my localhost. Also, I use shorturl.at to shorten the URL so I don't need to type the full ngrok address.
@yuripourre take a look at https://github.com/treblereel/elemental2-experimental/tree/webxr-hand-input
@treblereel it looks good and that was fast! It is very similar to what I did on my experiments, but I believe yours will generate the right types.
How can I generate the java classes from it?
run build.sh from the root and install webxr and webxr-hand-input from the maven folder.
i have no ar/vr devs so i can't test it ... sorry
@treblereel ok! I will try to test until next Sunday. Don't worry I can't thank you enough for making libGDX + webXR a reality. Your code was the ground work I needed to make my libGDX apps run into Oculus Quest!
@treblereel I think I will need some assistance with the build, I am close but some jars are missing.
The output of sh build.sh
is:
DEBUG: [...lots of lines]
WARNING: [...lots of lines]
INFO: Analyzed target //java/org/treblereel/gwt/elemental2/audio:libaudio-src.jar (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //java/org/treblereel/gwt/elemental2/audio:libaudio-src.jar up-to-date:
bazel-bin/java/org/treblereel/gwt/elemental2/audio/libaudio-src.jar
INFO: Elapsed time: 5.081s, Critical Path: 0.01s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
build.sh: line 78: jar: command not found
Seems that the only jar available is the audio. I am trying to understand what is missing, I will update here if I have any progress.
EDIT: I commented all the modules but webxr-hand-input and the jars were generated. I am going to test it now.
@treblereel I just hit the first blocker.
I am trying to port this example:
// This method expects a JsArray but XRHand extends JsIterable<String>
// So the line below cannot be called
frame.fillJointRadii(inputSource.hand.values(), radii)
I will keep trying and update here if I had some progress.
@treblereel awesome news, it's working!!!
I had to update XRHand and XRFrame (I also moved the code from WebXRHandInputXRFrame to XRFrame). I believe you know a more elegant way to update them.
The code I mentioned now looks like this:
if (!frame.fillJointRadii(inputSource.getHand().values(), radii)) {
console.log("no fillJointRadii");
return;
}
JsIteratorIterable<XRSpace> values = Js.uncheckedCast(inputSource.getHand().values());
if (!frame.fillPoses(values, refSpace, transforms)) {
console.log("no fillPoses");
return;
}
XRHand.java
@JsType(isNative = true, namespace = JsPackage.GLOBAL)
public interface XRHand extends JsIterable<JsMap.JsIterableTypeParameterArrayUnionType<XRHandJoint, XRJointSpace>[]> {
XRJointSpace get(String key);
@JsProperty
double getSize();
JsIteratorIterable<XRJointSpace> values();
}
XRFrame
@JsType(isNative = true, namespace = JsPackage.GLOBAL)
public interface XRFrame {
XRPose getPose(XRSpace space, XRSpace baseSpace);
@JsProperty
XRSession getSession();
XRViewerPose getViewerPose(XRReferenceSpace referenceSpace);
boolean fillJointRadii(JsIteratorIterable<XRJointSpace> jointSpaces, Float32Array radii);
@JsOverlay
default boolean fillJointRadii(XRJointSpace[] jointSpaces, Float32Array radii) {
return fillJointRadii(Js.<JsIteratorIterable<XRJointSpace>>uncheckedCast(jointSpaces), radii);
}
boolean fillPoses(JsIteratorIterable<XRSpace> spaces, XRSpace baseSpace, Float32Array transforms);
@JsOverlay
default boolean fillPoses(XRSpace[] spaces, XRSpace baseSpace, Float32Array transforms) {
return fillPoses(Js.<JsIteratorIterable<XRSpace>>uncheckedCast(spaces), baseSpace, transforms);
}
XRJointPose getJointPose(XRJointSpace joint, XRSpace baseSpace);
}
Thank you so much for your help and for your effort. I am planning to post something in gitter soon. I want to opensource my libGDX port but I am still figuring out the best way to do it.
@yuripourre always welcome ! join me in getter and follow our channels https://gitter.im/gwtproject/gwt and https://gitter.im/vertispan/j2cl
Hello @treblereel do you have any plans to add hand tracking to the WebXR classes?
How the JavaScript specification can be generated? By using WebIDL files?
I tried to add the classes manually but something is not mapped correctly so there are no hands available.
Thank you so much for your efforts! Thanks to you I could successfully make libgdx run into Oculus Quest.