Support Looking Glass portrait

[toji]

As if you didn't have enough to do. :)

Just got my Looking Glass Portrait and gave this a cursory test. Unfortunately the views aren't properly aligned (which is probably to be expected) so it just shows up garbled, even with the content fullscreen. Happy to help debug and fix this some time!

[kainino0x]

I am actually somewhat surprised this isn't working. Do you have the holoplay service running? If not you should see something like HoloPlay Service not running! in dev tools.

I can't find the site right now but there's an easy way to get the calibration data out of the holoplay service. I'll ask you when I find it, and then I can try testing with your calibration data.

[toji]

Yeah, that's how it comes out (rendering 2 views) and I haven't found a combination of setting yet that gets it to look sensible (except for 1 view). Also it's obviously expecting a landscape-oriented screen.

[toji]

There's a file on the device at LKG_calibration/visual.json that has some calibration values in it, though I'm not sure if they're the ones you need?

{
 "configVersion": "1.0",
 "serial": "LKG-PORT-XXXXX",
 "pitch": {
  "value": 52.563010659943416
 },
 "slope": {
  "value": -7.244674277559994
 },
 "center": {
  "value": 0.4016996953176948
 },
 "fringe": {
  "value": 0.0
 },
 "viewCone": {
  "value": 40.0
 },
 "invView": {
  "value": 1
 },
 "verticalAngle": {
  "value": 0
 },
 "DPI": {
  "value": 324.0
 },
 "screenW": {
  "value": 1536.0
 },
 "screenH": {
  "value": 2048.0
 },
 "flipImageX": {
  "value": 0
 },
 "flipImageY": {
  "value": 0
 },
 "flipSubp": {
  "value": 0.0
 }
}

[toji]

Oh, and I forgot to confirm that the service IS running. Version 1.2.0

[SoylentGraham]

Debugging, it IS loading a new calibration from the device, (and rebuilding fragshader) but still mis-aligned somewhere. For reference, this is the JSON from the [websocket] service ws://localhost:11222/

DPI: {value: 324}
center: {value: 0.5}
configVersion: "1.0"
flipImageX: {value: 0}
flipImageY: {value: 0}
flipSubp: {value: 0}
fringe: {value: 0}
invView: {value: 1}
pitch: {value: 52}
screenH: {value: 2048}
screenW: {value: 1536}
serial: "LKG-PORT-XXXXX"
slope: {value: -7}
verticalAngle: {value: 0}
viewCone: {value: 40}

[SoylentGraham]

Conversly, calibration from https://dhtk4bwj5r21z.cloudfront.net/HoloplayJS/examples/basic/index.html ws://localhost:11222/driver

DPI: {value: 324}
center: {value: 0.04356349632143974}
configVersion: "1.0"
flipImageX: {value: 0}
flipImageY: {value: 0}
flipSubp: {value: 0}
fringe: {value: 0}
invView: {value: 1}
pitch: {value: 52.58437728881836}
screenH: {value: 2048}
screenW: {value: 1536}
serial: "LKG-PORT-XXXXX"
slope: {value: -7.1081953048706055}
verticalAngle: {value: 0}
viewCone: {value: 40}

If I hardcode this calibration, it looks better, but still not right. Although... the FPS counter seems 3D & legible... (Really need to use holoplaycore.js to fetch the proper calibration as the data is all CBOR encoded, and a bit too much to hack in :)

[SoylentGraham]

This hacky-replacement will send a cbor encoded message, and get a cbor encoded message back with the correct calibration https://github.com/paroga/cbor-js (this is the package holoplaycore.js uses)

{
                      const ws = new WebSocket('ws://localhost:11222/driver', ['rep.sp.nanomsg.org']);
                      //const ws = new WebSocket('ws://localhost:11222/');
            ws.binaryType = 'arrayBuffer';
                      const timeout = setTimeout(() => {
                        console.error('HoloPlay Service running, but request timed out (no device attached?)');
                      }, 5000);
                      ws.onmessage = ev => {
                        clearTimeout(timeout);
                        try {
                console.log(`New Calibration from holoplay service`,JSON.parse(ev.data));
                          this.calibration = deepFreeze(JSON.parse(ev.data));
                        } catch (ex) {
                          console.error('HoloPlay Service running, but provided invalid calibration data:', ev.data);
                        }
                      };
                      ws.onerror = (ev) => {
                        clearTimeout(timeout);
                        console.error('HoloPlay Service not running!', ev);
                      };
                      ws.onclose = (ev) => {
                        clearTimeout(timeout);
                        console.error('HoloPlay Service connection closed', ev);
                      };
            ws.onopen = (ev) => {
                        clearTimeout(timeout);
                        console.error('HoloPlay Service connection connected', ev);

            //  init message is 
            //  { cmd:{ info: {}} }
            //  encodeded to cbor;
            const InitMessage_Cbor = [162, 99, 99, 109, 100, 161, 100, 105, 110, 102, 111, 160, 99, 98, 105, 110, 246];
            //  message is 32bit message id then cbor
            const InitMessage_Data = new Uint8Array(InitMessage_Cbor.length + 4);
            const InitMessage_View = new DataView(InitMessage_Data.buffer);
            InitMessage_View.setUint32(0, 0xdeadbeef );
                InitMessage_Data.set( InitMessage_Cbor, 4);
            ws.send(InitMessage_View);
                      };
                    }

[kainino0x]

I've updated the library to use holoplay-core instead of accessing the WebSocket directly in c1a3f669e113fdb51cbe81d3e605ef49df38e215. Hopefully this improves the situation?

As a reference, here's what one of the webxr-samples is expected to look like (default settings except "number of views to render" is cranked up to 145): https://photos.app.goo.gl/VYbfnF4qj9WJXCFP9 https://kai.graphics/holoplay-webxr/third_party/webxr-samples/immersive-vr-session.html

[kainino0x]

Found another issue (canvas does not resize properly), working on it

[kainino0x]

Canvas resize issue fixed in 613f763e48d6e2cab7b14578889b3caeb4d6fd8a

[SoylentGraham]

For other readers; this is working now with new calibration & canvas fix; https://twitter.com/soylentgraham/status/1396235828746199045

[toji]

Definitely looking better now! The final image still looks rather fuzzy on the webxr sample scenes but I think (and this is my inexperience with the hardware talking) that's just a side effect of the Looking Glass display for things that fall outside of a relatively narrow z range? The FPS counter, for example, show up rather cleanly.

https://user-images.githubusercontent.com/805273/119267159-47e1ab80-bba2-11eb-83d5-d7a027e4ae34.mp4

[kainino0x]

just a side effect of the Looking Glass display for things that fall outside of a relatively narrow z range

Yes, that's exactly it. If you happen to know WebXR demos that are light on background imagery, they would probably look better. (The first thing that comes to mind is Sketchfab, but it doesn't work with holoplay-webxr and it's hard to figure out why. Would really like to get that working...)

[kainino0x]

I suppose this bug is fixed now 🎉

[kfarr]

Confirming on my side this works as expected on the looking glass portrait. The key for me was playing with target fov, target size, # of views, target z, etc.

[kainino0x]

Awesome! Unfortunately most demos require different values for those, it's tricky to get in the right spot.