search

DLR-RM / 3DObjectTracking

Algorithms and Publications on 3D Object Tracking
MIT License
713 stars 129 forks source link

Initialization workflow #13

Closed samuelmorais closed 2 years ago

samuelmorais commented 2 years ago

Hi,

I ported your great code to iOS and it is running very well, but I have a doubt about the initialization process...

I developed an initialization based on UI where the user can see a reference object on the screen...

But the new users simply don't know how to match the reference 2D object with the real object, although expert users can do it very quickly.

Do you have a suggestion on how to achieve an easy to use initialization UX via user interface or a code that helps with the initialization?

Another question:

Do you have a way to detect if the system is tracking or if it lost tracking? Kind of a variable or function that can determine that the tracking is lost or not?

manuel-stoiber commented 2 years ago

Hi,

currently, we are mostly focusing on tracking and do not provide any advanced automatic detection. Note, however, that many OS solutions exist for this (e.g. https://github.com/DLR-RM/AugmentedAutoencoder). In our most recent publication ICG, we provide an easy method, where the user has to specify four points in the image to initialize the object pose.

Right now, we also do not have an OS tracking loss detection. However, this might change in the future.

samuelmorais commented 2 years ago

Thanks for you answer. About the automatic detection, I already used the Augmented Autoencoder through the ISAAC SDK, but I am targeting mobile devices and I didn't find a way to run the inference on iOS and Android (sending the images to the server would be not viable).

So the RBGT and SRT3D where more useful, although now I am struggling on how to provide a User-friendly initialization interface that can detect the object.

Anyway, I will check the ICG initialization and if it doesn't use RBG-D to create the four points I think I can copy the code and integrate it with RBGT or SRT3D.

Congrats for this work.

FaiyazMeynikara commented 1 year ago

@samuelmorais . Hey am about to start porting this to Unity3D,. Typically a android/IOS plugin and a C# wrapper for Unity3D. Any quick start guide to refer?

samuelmorais commented 1 year ago

Hey, You should try to create custom cameras and send pixel bytes from Unity to the c++.

Em seg., 4 de set. de 2023 às 11:04, FaiyazMeynikara < @.***> escreveu:

@samuelmorais https://github.com/samuelmorais . Hey am about to start porting this to Unity3D,. Typically a android/IOS plugin and a C# wrapper for Unity3D. Any quick start guide to refer?

— Reply to this email directly, view it on GitHub https://github.com/DLR-RM/3DObjectTracking/issues/13#issuecomment-1705330931, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEU5J3G2L5FW4QQWZITP2WLXYXNYLANCNFSM5K2PSEKA . You are receiving this because you were mentioned.Message ID: @.***>

FaiyazMeynikara commented 1 year ago

Understood. Thanks for the heads-up. Didn't realize Unity supports Native C++ libraries.

samuelmorais commented 1 year ago

Yeah, you can do it for iOS and Android.

Em seg., 4 de set. de 2023 às 12:18, FaiyazMeynikara < @.***> escreveu:

Understood. Thanks for the heads-up. Didn't realize Unity supports Native C++ libraries.

— Reply to this email directly, view it on GitHub https://github.com/DLR-RM/3DObjectTracking/issues/13#issuecomment-1705436121, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEU5J3DENSX7XCY3VZ5JXY3XYXWKRANCNFSM5K2PSEKA . You are receiving this because you were mentioned.Message ID: @.***>

sivatd007 commented 3 months ago

I ported M3T code into Android Studio facing a compilation error with the GLFW library, Is there any replacement for that else any setup guidance for that

stefan981 commented 2 months ago

@FaiyazMeynikara Were you able to get it run on iOS with Unity?

samuelmorais commented 2 months ago

@stefan981 I was able to get it working. What I can say is that, when it tracks the object it keeps tracking if you don't move the object too quickly. And you need to match the initial pose at the starting of the session. But if you loose tracking, it starts searching for the lost pose and doesn't recover anymore unless you implement a "start again" command.

FaiyazMeynikara commented 2 months ago

@FaiyazMeynikara Were you able to get it run on iOS with Unity?

Unfortunately I didn't attempt that as the project is on hold. I was able to compile it as dll and able to run on unity editor and as exe build. As samuel mentioned , it works correctly if initial pose set and if I move little too fast , it crashes and force closes.