search

Unity-Technologies / arfoundation-samples

Example content for Unity projects based on AR Foundation
Other
3.02k stars 1.12k forks source link

OcclusionMeshes problem:All the 3D models disappera beyond the depth limit (30 meterts) #979

Closed yuhacrows closed 6 months ago

yuhacrows commented 2 years ago

Hi ARF

I use OcclusionMeshes sample.And touch screen to shoot red balls.All these red dots will disappera at about 20 meters. If I follow these red balls.They will not disappera.(in ARF 4.2.3,red balls will disappera at 8 meters) I want to show some 3D model out of 20 meters.Such like a long coin line for 50 meters. I know that depth limite is 20 meters,but why disappera 3d model out of the depth range. If phone recognition it's long distance depth out 20 meters，it should show the 3d model,not disappera it.

So how can I fix this?I want to show 3d model out 20 meters.

Sorry for my bad english. Thank you Yuha

andyb-unity commented 2 years ago

Take a look at the AR Camera GameObject, a child of the ARSessionOrigin. (Or alternatively, Camera.main via scripting) The default Far Clipping Plane value on the Camera component is 20 meters, which you might want to increase. https://docs.unity3d.com/ScriptReference/Camera-farClipPlane.html

yuhacrows commented 2 years ago

Take a look at the AR Camera GameObject, a child of the ARSessionOrigin. (Or alternatively, Camera.main via scripting) The default Far Clipping Plane value on the Camera component is 20 meters, which you might want to increase. https://docs.unity3d.com/ScriptReference/Camera-farClipPlane.html

Thankyou,but it's not work.Acturelly,I have already change this value to 200 before I post this problem.But the red balls still disappera at about 25-30 meters. I think it's depth map's max value problem.If the distance >20~30meters,the depth image will show a pink color. If turn off the Temporal Smoothing of the depth map.And rotate phone fast. It will show 3D model>30meters at once,because the phone has‘t calculate the depeh.And the the depth image show a dark blue color. After one second,it calculate the depth,depth image turn to pink color.And the 3D>30meters model disappera.

So maybe it'a depth generate problem.If distance >25-30meters,it will give a value set it at 25-30.The max dpeht value. I think just don't limit the max value.If distance >25-30meters,set it to a unidentified distance value.Same as the dark blue color in the depth image.This should fix the problem.

Sorry for my bad Enlgish.

yuhacrows commented 2 years ago

I can't not re open this.

yuhacrows commented 2 years ago

@andyb-unity @ankur-unity Hello~?

DavidMohrhardt commented 2 years ago

Could you please file a bug? If you haven't modified the sample then just include the project otherwise please try to reproduce it with the unchanged ARFoundation sample. https://unity3d.com/unity/qa/bug-reporting

Some general comments. Depending on your provider (ARCore or ARKit) there are hard limitations to the maximum reportable depth. On ARCore specifically has a maximal span of 8-65 meters depending on which version of ARCore and your device's capabilities. On ARKit it's limited by the LiDAR sensor which Apple has not specified the maximum distance of.

yuhacrows commented 2 years ago

Could you please file a bug? If you haven't modified the sample then just include the project otherwise please try to reproduce it with the unchanged ARFoundation sample. https://unity3d.com/unity/qa/bug-reporting

Some general comments. Depending on your provider (ARCore or ARKit) there are hard limitations to the maximum reportable depth. On ARCore specifically has a maximal span of 8-65 meters depending on which version of ARCore and your device's capabilities. On ARKit it's limited by the LiDAR sensor which Apple has not specified the maximum distance of.

Hi

Thank for reply. I use origin ARFoundation 5.0 sample.Update to pre12. Change nothing.Just add some cube infront of the camera.Each 5 meters set one cube.(Use Simple Occlusion scene) Then run the sample on phone,see which distance cube will disappera. The result is 20~30 meters. Test in samsung note10+ & iphone12pro with LiDAR & xiaomi mix2s & Oneplus 9. (In ARFoundation 4.3,the result is 8 meters.)

And"ARCore specifically has a maximal span of 8-65 meters" It's not 65 meters.It's 65 feets.Google made a mistake.Google use 65 feet not 65 meters in the video of arcore 1.3.1.

The problem is depth map. If distance > max depth limit,like 100 meters.The depth map can't recognized the real distance. Then it will use the max distance value "20 ~ 30"meters to replace the real distance(100 meters). Then the occlusion shader will occlusion the 3d object > 20 ~ 30meters.

I think change the depthmap generate rule can fix this problem, If the distance > limit value.Just give an unrecognized distance value. Not give a limit distance value to depthmap. Then the occlusion shader will not occlusion the 3D object > limit distance.

ankur-unity commented 2 years ago

(In ARFoundation 4.3,the result is 8 meters.)

There is no AR Foundation v4.3. If you're referring to v4.2.3 then yes, the maximum depth value is ~8 meters in that version. The range was increased in ARCore SDK 1.31 to allow depth values up to 65 meters. AR Foundation 5.0.0-pre.12 updated the ARCore SDK to v1.31 and provides the new 65 meters depth range.

It's not 65 meters.It's 65 feets.Google made a mistake.

The depth accuracy reduces significantly after ~20-25 meters which translate to 65-82 feet. See Google's ARCore documentation .

Google use 65 feet not 65 meters in the video of arcore 1.3.1.

Can you provide a link to the video with timestamp where Google uses 65 feet instead of meters?

If distance > max depth limit,like 100 meters.The depth map can't recognized the real distance. Then it will use the max distance value "20 ~ 30"meters to replace the real distance(100 meters).

According to Google's documentation, the pixels beyond the supported range of 65 meters will have 0 value and not a default distance. See Google's ARCore documentation for Frame.acquireRawDepthImage16Bits.

yuhacrows commented 2 years ago

Hi ,Thanks for reply

1 Yes It's ARF 4.2.3 2 https://www.youtube.com/watch?v=PM5rl4z9mto at 7:43 3 Yes,But Iin ARF sample,the depthmap is use the 20 25meters for the max depth value,So the depth occlusion will disappera all the 3d object from 20 25meters.

Can you try?Run the sample,depth ----Simple Occlusion,tap screen and shoot the red balls.See how far the redballs disappera.(the red ball's speed is one second one meter,just count the second,you can know the distance. ) (Change the Far Clipping Plane value on the Camera)

Really thank you.Hope can help me to fix this problem. This really a big problem for our project.We are doing outside VPS AR. And we also need this Depth Occlusion on IOS.So not only the arcore.

yuhacrows commented 2 years ago

And there is another thing,Xiaomi mix2s doesn't support depth in the arcore device list. But it still can run the depth in ARF.

tdmowrer commented 2 years ago
  • Pixels without a valid depth estimate have a pixel value of 0 and a corresponding confidence value of 0.

It seems to me that if the confidence is zero, then the shader should ignore that depth value. They are basically saying "we have no depth data for this pixel" so there should be no occlusion.

yuhacrows commented 2 years ago
  • Pixels without a valid depth estimate have a pixel value of 0 and a corresponding confidence value of 0.

It seems to me that if the confidence is zero, then the shader should ignore that depth value. They are basically saying "we have no depth data for this pixel" so there should be no occlusion.

Yes! no depth data for no occlusion. Shader should ignore that depth value>20.This will fix all the problem.Even in the smooth mode.

DavidMohrhardt commented 2 years ago

As a workaround you may be able to copy the shaders from the Library/AR Provider Package/Assets/BackgroundShader.shader and modify the function ConvertDistanceToDepth to return 1.0 for d values of 20 or above then make it a custom material for your scene while we look into this issue.

See https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@5.0/manual/index.html#ar-camera-background for some more info on custom background materials.

yuhacrows commented 2 years ago

As a workaround you may be able to copy the shaders from the Library/AR Provider Package/Assets/BackgroundShader.shader and modify the function ConvertDistanceToDepth to return 1.0 for d values of 20 or above then make it a custom material for your scene while we look into this issue.

See https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@5.0/manual/index.html#ar-camera-background for some more info on custom background materials.

It Works!!! Thank all of you.I will share some video these days.

Rajuuu10 commented 2 years ago

@yuhacrows @DavidMohrhardt Will you please share the video or some details on the modification you did for the solution to the issue? I tried but I can not figure it out.

I am facing below posted issue: https://forum.unity.com/threads/ar-object-disappear-at-some-distance-though-i-set-ar-camera-far-clipping-plane-value-to-5000.1296225/#post-8275620

yuhacrows commented 2 years ago

Yes,I will share some video this week.

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

switchstance-devs commented 1 year ago

@yuhacrows - do you have that video/workaround please?

enginetla commented 1 year ago

As a workaround you may be able to copy the shaders from the Library/AR Provider Package/Assets/BackgroundShader.shader and modify the function ConvertDistanceToDepth to return 1.0 for d values of 20 or above then make it a custom material for your scene while we look into this issue.

See https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@5.0/manual/index.html#ar-camera-background for some more info on custom background materials.

I managed to find the shader, but I do not know how to change it correctly to fulfill this point: modify the function Convert Distance To Depth to return 1.0 for values of 20 or above then make it a custom material for your scene while we look into this issue.

could you please help me?

cheuaheichan commented 1 year ago

@yuhacrows Can you please share your solution?

I can't find the Library/AR Provider Package/Assets/BackgroundShader.shader

cheuaheichan commented 1 year ago

Okay, nevermind my comment above. I now found out how to do it. Just for everyone after me, here's how I did it.

  1. Find the BackgroundShader.shader:

It's under Packages > Google ARCore XR Plugin > Assets > Shaders for Android and Packages > Apple ARKit XR Plugin > Assets > Shaders for iOS

  1. Make a duplicate of this shader
  2. Open the file in Visual Studios or similar
  3. Go to the function float ConvertDistanceToDepth (float d), for me it was line 93
  4. Add these two lines for code to apply the suggested fix

if (d > 20.0F) return 1.0f;

so it looks like this:

`
float ConvertDistanceToDepth(float d) { if (d > 20.0F) return 1.0f;

            d = _UnityCameraForwardScale > 0.0 ? _UnityCameraForwardScale * d : d;

            float zBufferParamsW = 1.0 / _ProjectionParams.y;
            float zBufferParamsY = _ProjectionParams.z * zBufferParamsW;
            float zBufferParamsX = 1.0 - zBufferParamsY;
            float zBufferParamsZ = zBufferParamsX * _ProjectionParams.w;

            // Clip any distances smaller than the near clip plane, and compute the depth value from the distance.
            return (d < _ProjectionParams.y) ? 1.0f : ((1.0 / zBufferParamsZ) * ((1.0 / d) - zBufferParamsW));
        }

`

  1. In Unity create a new Material and choose Shader -> Unlit/ARCoreBackground
  2. In AR Camera Background check "Use Costum Material" and attach the above newly created Material to Custom Material
  3. Profit

I couldn't attach the BackgroundShader.shader here :(

Cheers!

yuhacrows commented 1 year ago

Yes,And I rember that problem has already fix one year ago.In AR FOUNDATION Update detail.

发件人: cheuaheichan @.> 发送时间: 2023年8月18日 10:39 收件人: Unity-Technologies/arfoundation-samples @.> 抄送: yuhacrows @.>; Mention @.> 主题: Re: [Unity-Technologies/arfoundation-samples] OcclusionMeshes problem:All the 3D models disappera beyond the depth limit (30 meterts) (Issue #979)

Okay, nevermind my comment above. I now found out how to do it. Just for everyone after me, here's how I did it.

  1. Find the BackgroundShader.shader:

It's under Packages > Google ARCore XR Plugin > Assets > Shaders for Android and Packages > Apple ARKit XR Plugin > Assets > Shaders for iOS

  1. Make a duplicate of this shader
  2. Open the file in Visual Studios or similar
  3. Go to the function float ConvertDistanceToDepth (float d), for me it was line 93
  4. Add these two lines for code to apply the suggested fix

if (d > 20.0F) return 1.0f;

so it looks like this:

` float ConvertDistanceToDepth(float d) { if (d > 20.0F) return 1.0f;

        d = _UnityCameraForwardScale > 0.0 ? _UnityCameraForwardScale * d : d;

        float zBufferParamsW = 1.0 / _ProjectionParams.y;
        float zBufferParamsY = _ProjectionParams.z * zBufferParamsW;
        float zBufferParamsX = 1.0 - zBufferParamsY;
        float zBufferParamsZ = zBufferParamsX * _ProjectionParams.w;

        // Clip any distances smaller than the near clip plane, and compute the depth value from the distance.
        return (d < _ProjectionParams.y) ? 1.0f : ((1.0 / zBufferParamsZ) * ((1.0 / d) - zBufferParamsW));
    }

`

  1. In Unity create a new Material and choose Shader -> Unlit/ARCoreBackground
  2. In AR Camera Background check "Use Costum Material" and attach the above newly created Material to Custom Material
  3. Profit

I couldn't attach the BackgroundShader.shader here :(

Cheers!

― Reply to this email directly, view it on GitHubhttps://github.com/Unity-Technologies/arfoundation-samples/issues/979#issuecomment-1683717472, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACCQZKJWFQHYO6ACK6YI7MLXV5A5ZANCNFSM5W4OOHJA. You are receiving this because you were mentioned.Message ID: @.***>

ankur-unity commented 6 months ago

Closing this since there is a working workaround as discussed above and the thread has been inactive for some time. Feel free to re-open if there are additional questions on this topic.