search

Unity-Technologies / arfoundation-samples

Example content for Unity projects based on AR Foundation
Other
3.07k stars 1.15k forks source link

How to use Ar foundation camera with openCV #682

Closed Ivan547-arnus closed 3 years ago

Ivan547-arnus commented 3 years ago

i was using webcametexture and my computer vision algorith work however i need functions of ar foundation in same project so i want adapt my project with camera from ar foundation, any idea?

tdmowrer commented 3 years ago

If you want the raw camera pixels on the CPU, there's an API for that and a sample scene.

Ivan547-arnus commented 3 years ago

Yes, i want get texture2D from ar foundation and use in open cv to show in Raw image, i´ll see the sample scene

Ivan547-arnus commented 3 years ago

I can use the cpu image but the texture2D is rotated so to solve this i used opencv rotate but the image is a bit streched, any idea to perserve aspect?

tdmowrer commented 3 years ago

ARFoundation's API just provides the raw pixel data, with a few limited transformations (like mirroring).

Your question concerns the OpenCV API, which is out of scope for this issues page. Sorry; I can't tell you why a third-party library does something unexpected. OpenCV does have a pretty extensive community, though. Have you tried their forum?

Ivan547-arnus commented 3 years ago

Ok, thanks i should see in openCV forum

Ivan547-arnus commented 3 years ago

ok, everything work fine but the app crash after a few minutes of use, this happens even if i dont use computational vision algorithm

Ivan547-arnus commented 3 years ago

@tdmowrer can you help me? please

mdurand42 commented 3 years ago

@tdmowrer can you help me? please

If you can share a stack trace of log information as part of a Unity Issue Tracker report then we should be able to tell whether the crash is in application code. Please log an issue. https://unity3d.com/unity/qa/bug-reporting

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

yosun commented 3 years ago

been able to get OpenCVForUnity to work with ARFoundation/ARKit since at least 2018 (haven't tried recently)

Ivan547-arnus commented 3 years ago

been able to get OpenCVForUnity to work with ARFoundation/ARKit since at least 2018 (haven't tried recently)

@yosun wich was your solution to rotate camera

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Weasy666 commented 3 years ago

@yosun @Ivan547-arnus did you get ArUco markers to work with ArFoundation CpuImage? I am able to detect the markers with OpenCvForUnity, but i can not seem to get the OpenCv pose of the marker correctly converted to a Unity pose. When i use the OpenCv pose and convert it from right handed to left handed, the on-screen position and rotation of my marker-cube is not aligned to the marker in the image.

Ivan547-arnus commented 3 years ago

Hi @Weasy666 you should rotate cpu image before of you opencv algorithm otherwise your result pose not work however i am not using ArUco marker

Weasy666 commented 3 years ago

Thanks for the fast answer. You are right, found my problem shortly after asking here. A tip for others, when using Utils.fastMatToTexture with the parameterflip=true, the image will not just be flipped for the texture, but the src Mat will also get flipped.