auto generated C# bindings / Toolchains iOS/Android

[roscopecoltran]

Hi,

Hope you are all well !

I am actually working on some C++ libraries, like Dlib or OpenCV, and I wanted to auto generate unsafe bindings for some computer vision libraries to build native plugins for Unity3D.

After a couple of researches, I found your great work and some questions popped out of my brain :-)

Context: I wanted to test this example, dnn_mmod_dog_hipsterizer.cpp, from dlib, in order unity3d to leverage the WebCamTexture class. I saw a module in the asset store using dlib but I wanted to test ccv and some other CMake driven projects. So, I am looking for something scriptable and more flexible for my studies.

So, in a nutshell, I would like to auto-generate bindings from some C++ libraries and allow interoperability and build these projects with CMake, with polly, a collection of toolchains to cross-platform build and test these projects..

I found CppSharp and still trying to find out the best way to get something, like FFmpeg.AutoGen, also using CppSharp.

Ultimately, I would like to create a boilerplate like this project testing cross-language interactions (link). To make it more flexible, I was thinking to wrap all these commands inside a docker container with onbuild instructions. Even if you cannot compile xcode projects from a docker container, it would be great for Android, Linux or Windows builds.

Questions:

bindings related:

1. How did you proceed for aruco-unity like Calib3d.cs ? Did you write all the bindings yourself ? (if yes, that's a lot of work :-) )

2. Did you try to use Swig or CppSharp for the aruco-unity project ?

3. How would you see a much more structured and semi-automated flow to build native modules, based on Cmake driven projects, for Unity3d ?

cmake related:

3. Would aruco-unity work for iOS or Android if we provide a cmake toolchain ?

Thanks in advance for your input on my quest :-)

Wish you a great week !

Cheers, Richard

[NormandErwan]

Hi,

I had an equivalent context than you: there is some AR modules in the asset store but I wanted to use the OpenCV's ArUco module in Unity. And I wanted a free license module. That's why I started this project.

To answer your questions:

bindings related:

1. I wrote all the bindings by myself. First I just wanted to test the ArUco library on Unity. At this time, I didn't know anything about bindings, so I started with manual bindings in order to learn. And it worked pretty well as I expected so I continued on to finish the project. Retrospectively I may have used Swig or CppSharp, because yes it's a lot of work!
2. I haven't take the time to try Swig or CppSharp yet! I think I should, my manual bindings are far from perfect. For example I should use IDisposable interface to manage the memory, there is no tests, or there may be memory leaks with the actual bindings. On the other hand, the core of ArUco Unity is almost finished (port the features of ArUco for Unity for Windows).
3. It could be very interesting. I wanted first with this project to make ArUco available for Unity for Windows. Then port the project to other desktop and mobile platforms. I think it could be a great idea to use Hunter and Polly.

cmake related:

1. Yes this is possible (see this ArUco port for Unity for Android, and OpenCV for Unity works for Android and iOS). When the port for Windows will be finished, I will port to Android, iOS, Linux and Mac OS. But I'm wondering of the performances on mobile with the current project's architecture.

I don't know anything about docker, but it may be interesting to use. Tell me if you test ccv and Dlib on Unity, I'm interested about it. If you want to use WebCamTexture with these libraries, take a look ArucoCameraWebcam.cs and ArucoCamera.cs to put the WebCamTexture to a more easier to use Texture2D with which you can use GetRawTextureData() and LoadRawTextureData() to get and set pixels as an array of bytes.

Does that answer well your questions?

Best Erwan