Render distorted image correctly on screen.

[cdytoby]

Detail

The following parameters or properties must be correct

• Distortion
• Camera view parameters
• Camera eye distance

And find a way to verify if these are correct.
One possible approch is, create an exact same scene as Google Cardboard Demo, and compare it with our demo.

[cai-mobfish]

According to old cardboard,

• "\Assets\GoogleVR\Scripts\GvrEye.cs"
• "\Assets\GoogleVR\Scripts\VRDevices\BaseVRDevice.cs"

Camera eye position is decided by cardboard viewer lense separation, which is also InterLensDistance. No rotation is applyed.

The new cardbaord introduced some other parameters, it should also be considered.

InterLensDistance: Horizontal distance between optical center of the lenses. This is a required field for distortion rendering, and must be positive

[cai-mobfish]

My test package, it's the same as in the demo folder.

test.zip

It seems InterLensDistance is already enough, but more test is needed.

[cai-mobfish]

Note:

For Matrix, both Unity and GLKit(OpenGL) are column major, but GLKit may have different array order for matrix. Detail here: https://stackoverflow.com/questions/9688509/how-is-glkits-glkmatrix-column-major

For Coordinate, Unity is left handed, GLKit is right handed.

However, projection matrix seems can be used without modification and without issue. So I believe array order is not a problem.

[cdytoby]

What I did:

There are 2 different apps on a same device: one is this unity project (A), another is a test project (B) with exactly the same scene setup, and current unity vr with cardboard option. Package is available above in comment. The goal is, to test if this unity project have exact the same view as the old unity vr with cardboard.

Both app will need to scan the same qr code.

There are currently 2 issues here:

1. Looks like either FOV or eye position z axis is incorrect. Because the image in a single eye in the A shows a little more than B. But the object-eye-alignment seems correct. For example when the cube is aligned on left eye's left edge, and the cube position on the other eye are the same between A and B
2. On some devices, eye mesh distortion has incorrect scale. I need more test on this but these are the screenshots I got from my collage (iphone 11 pro), both apps have the same qr code scanned: B:
   A:
   

But it's not the case on my iPad Air (although it doesn't support cardboard technically), both app shows exactly the same distortion mesh at the same position. I have no idea if this is intended within the new sdk or the implementation is just wrong. Need more test on this.

I suspect this has something to do with eyeFromHeadMatrix, because it's not used currently, I implemented it according to the file https://github.com/googlevr/cardboard/blob/master/hellocardboard-ios/HelloCardboardRenderer.mm, the lens distance is far more away than intended I believe, so my implementation is wrong. Some other parameters in DeviceParams may also need to be considered.

Also note, I noticed that there maybe a bug within the old Cardboard, after qr code is scanned, not all properties are applied, with the same qr code, the result view maybe different after another scan. I'm not sure about this.

[cdytoby]

OK I can confirm that the old cardboard in unity and new cardboard (native sdk) has different view result with the same qr code.

This is the result with the same device and same qr code. We can already see the black edge are different.

[cai-mobfish]

https://github.com/googlevr/cardboard/blob/master/sdk/lens_distortion.cc

According to this file, eye_from_headmatrix is nothing more than inter_lens_distance offset.

[cai-mobfish]

@rohantipare

• Please give me feedback how the views look like on your side.
• We have already used CardboardMesh and projection_matrix, they're probably related to DeviceParams in internal source code, InnerLensDistance is used in eye_from_head_matrix and I used InnerLensDistance directly. If possible, can you check in the Cardboard SDK, and see if there are any parameters in DeviceParams, which is relevant to rendering and not yet used in the code above mentioned?

My progress for now: with InnerLensDistance the view should be correct. Old and new cardboard sdk do render things differently so a slightly difference is expected. I want to make sure I don't miss anything.

[rohantipare]

@cai-mobfish I was able to reconstruct the scene from HelloCardboard in unity. I disabled the camera movement in both the builds. I have tested them for 2 variations of device parameters. Here are the screenshots:

HelloCardboard screenshots:

Unity screenshots without using InterLensDistance to modify left and right camera positions- (Existing code): - Slight variation in lateral position as compared to 'HelloCardboard'

Unity screenshots while considering InterLensDistance: - Exactly matches the 'HelloCardboard' Screenshots.

I think, using the InterLensDistance to modify left/right camera positions has fixed the issue. Device used: iPhone 6s - iOS 13

[cai-mobfish]

@rohantipare Great! then I'll take this ticket again, do some refactor and test, then this issue is done.

If possible, can you export this scene/asset or prefab into unitypackage and upload it? This is valuable for testing purpose.

Or push it with the code into a new branch.

[rohantipare]

@cai-mobfish I have pushed the scene to branch 'RenderValidation'. 'DemoScene' is modified and the Camera movement is disabled.

[cai-mobfish]

Merged into public-dev

This ticket can be closed. But new ticket is needed to add reconstructed scene from google cardboard.