InstantPreviewInput creates a picking problem

[DomDumont]

Hi,

I'm trying to pick an object previously instantiated in my AR scene.

I'm using the "Unity way" with FirstPersonCamera.ScreenPointToRay and Physics.Raycast. Everything works fine when I really build the app (Build and run in unity).

But when I use the "instant preview" the picking is not working anymore.

I have this line in my code : using Input = GoogleARCore.InstantPreviewInput;

is there anything thing else to do ?

Thanks

[chaosemer]

Can you please give more context? How are you calling ScreenPointToRay?

[DomDumont]

Sure,

at the top of my file i have

#if UNITY_EDITOR
using Input = GoogleARCore.InstantPreviewInput;
#endif

In my Update function I have :

Ray ray = FirstPersonCamera.ScreenPointToRay(touch.position);
RaycastHit hit;
if (Physics.Raycast(ray,out hit)) {
        Debug.Log("hit something");
}

When I deploy to my phone, this code works.

With instantpreview, the ray seems to have a bad orientation. If i click on the left or on the right of my object sometimes I hit it (by pure luck). It acts as if the ray (or the objects) are not reoriented correctly (throught anchors, etc...).

[chaosemer]

How are you getting the touch? From Input.GetTouch(0), I assume?

This sounds like a bug. Could you please share a minimal repro project? That will help us debug what is going on.

Thanks!

[DomDumont]

Yes I’m getting the touch from Input.GetTouch(0). Sure, I will give you a repro project asap ( probably on monday ) because I don’t have my computer with me this week-end. Thanks

[DomDumont]

Hello, Here's the minimal project : https://github.com/DomDumont/ARPickingTest

I've just modify your HelloAR sample. Now the controller has 2 states (SPAWNING and PICKING) and alternately goes from one to another. you spawn andy then you try to pick it. If the pick is a success, andy is removed from the scene and you go back to the spawning phase again.

[chaosemer]

Thank you! I've sent the sample to the Instant Preview team to take a look.

[gstanlo]

There's currently a bug with the incoming touch position in InstantPreviewInput.cs. The incoming nativeTouch position is going to be in the coordinates of your phone's resolution, e.g. (1080, 1920). However, the Y value is being transformed by Screen.height - nativeTouch.Y, which is assuming your preview window is the same resolution as your device.

You can transform this correctly with some maths, careful to acknowledge that another Screen transform exists in raycast code here: https://github.com/google-ar/arcore-unity-sdk/blob/9f8d5a556dd0f54aa2eeaa27c1f4ff82b35c923e/Assets/GoogleARCore/SDK/Scripts/Frame.cs#L104

You can also set the screen size to match that of your phone and the maths should work out.

To get the proper X, Y for Unity, you could use something like: var x = touch.position.x * Screen.width / deviceWidth; var y = touch.position.y * Screen.height / deviceHeight;

[DomDumont]

Ok, I understand. I can work now by fixing my unity resolution to 1080x1920 ;-) Thanks !