search

raulmur / ORB_SLAM2

Real-Time SLAM for Monocular, Stereo and RGB-D Cameras, with Loop Detection and Relocalization Capabilities
Other
9.27k stars 4.69k forks source link

Unity3D native plugin #243

Open roscopecoltran opened 7 years ago

roscopecoltran commented 7 years ago

Hi guys,

Hope you are all well !

I was wondering if there is any plan to wrap ORB SLAM2 as a cross-platform native plugin c++ (managed or un-managed) for Unity3D. That would be awesome as there is plenty of native camera plugins available for iOS and Android, and would make development easier and faster ?

Have a good week !

Cheers, Richard

roscopecoltran commented 7 years ago

Hi,

Hope you are all well and thanks for the link ! :-)

To give you a better overview, I tried to wrap it into a static/shared library through cmake with hunter and polly. You can check a cross-platform project with OpenCV3 + OGLES_GPU, using the sniper/polly, at this repo url https://github.com/headupinclouds/gatherer.

Hunter makes it easier to build dependencies per version and Polly to setup the proper toolchain for each platforms; https://github.com/hunter-packages.

Ideally, it would work like this example from OpenCVForUnity https://github.com/EnoxSoftware/OpenCVForUnity/blob/master/OpenCVForUnity/Samples/ArUcoSample/ArUcoTexture2DSample.cs#L155

Unity Workflow

  1. Instantiate a detector object (using System.Runtime.InteropServices)
  2. for each frame updates, process the input matrix (provided by OpenCVForUnity)
  3. process the returned transform values and apply them to a set of game objects.

Unity Odometry plugin wrapping

  1. Generate a shared/static library for several platforms from CMakeLists (Hunter + Polly) with a script (eg. Mixpanel native plugin for Android, iOS, OSX, Windows and Linux) a. would provided a shared library (.so) for Android, Unity Editor b. would provided a static library (.a) for MacOSX and iOS, and the bundle files c. would provided all DLLs for windows platforms
  2. Generate wrapper around ORB_SLAM2 native detector methods and functions

Ps. I wanted to try https://github.com/mono/CppSharp and https://github.com/mono/embeddinator-4000 to make a kind of semi-automated wrapper for cpp native plugins.

With that we could build a project for any kind of platform and have something more easy to develop, I think. Maybe, I am wrong...

Any feedback is welcomed :-)

Have a good day !

Cheers, Richard

roscopecoltran commented 7 years ago

Are u french ?

roscopecoltran commented 7 years ago

martin sounds french, sorry about that :-)

Just to share some examples of ORB SLAM2 + Unity, I saw on youtube: https://www.youtube.com/watch?v=rnODCkjPtq4 https://www.youtube.com/watch?v=XVhdpCmyQtw https://www.youtube.com/watch?v=qQ9tkOiPAhQ

roscopecoltran commented 7 years ago

He made an awesome job :-) And we have exchanged a quick email today

AhmadBehzadi commented 7 years ago

@roscopecoltran, @Martin20150405 Hi, Got any result with that effort?

I'm trying to add a simple AR text in this iOS sample, but I'm very new to that and I don't know how to detect position (XYZ) of the real world object when user touches some where on screen(X,Y of mobile screen). Any idea?

carlosfarinhas commented 7 years ago

@roscopecoltran any news about your development on unity 3d native plugin, like gaoxiang?

carlosfarinhas commented 7 years ago

I was able to build on windows with VS15, livecam/webcam, and ORBvoc.bin! But now, the problem is wrap ORB SLAM2 as a cross-platform native plugin c++ (managed or un-managed) for Unity3D...

carlosfarinhas commented 7 years ago

@roscopecoltran And I have seen already the stereo version of it! https://github.com/VisionerTech/ORB_SLAM2_Unity

How to go from stereo render to mono (just using mobile webcam)?