search

ros2-dotnet / ros2_dotnet

.NET bindings for ROS2
Apache License 2.0
136 stars 54 forks source link

[Feature] Support Unity Editor #81

Open ooeygui opened 2 years ago

ooeygui commented 2 years ago

This feature is being implemented by Azure Edge Robotics. Payload incoming.

esteve commented 2 years ago

@ooeygui as part of #82 , one of the goals of ros2-dotnet was to have support for Unity, which was added in order to support Hololens. So I don't know what's missing here, can you please elaborate? Thanks

ooeygui commented 2 years ago

Unity Editor has some interesting behaviors that need to be factored into ROS2.net and the unity package. ROS2 itself needs to be enlightened to this fact (which we have a pending change up stream for), but also how load library works in ROS2.net. When packaging a component for Unity, metafile declare where individual binaries will be deployed - but they aren't in the execution path. LoadLibrary doesn't function correctly.

Hololens has a similar problem - LoadLibrary itself is missing in the API set. You need to use LoadPackagedLibrary, which loads it from the UWP package. (A similar fix is made in ROS2 itself).

(On top of this, the Unity native compiler will optimize out Reflection only methods (it doesn't know they are going to be called), so you need to add a link.xml to prevent the optimizer from destroying the ability to bind to ROS messages. This hasn't been addressed in ROS2.net documentation yet)

esteve commented 2 years ago

Hololens has a similar problem - LoadLibrary itself is missing in the API set. You need to use LoadPackagedLibrary, which loads it from the UWP package. (A similar fix is made in ROS2 itself).

Yes, and that's why rcldotnet_common has this code to use LoadPackagedLibrary instead of LoadLibrary for UWP.

Moreover, https://youtu.be/lQXtoK3w5X8 and https://youtu.be/mGTKNB-Iog0 show using ros2-donet in a Unity app from a Hololens.

Can you elaborate further what's needed? ros2-dotnet always had support for UWP, Unity and the Hololens, it was the very reason it was created. You can find more info about it in http://www.robocity2030.org/2018/06/18/rosin-grant-urjc/ and the talk I gave at ROSCon 2018 about ros2-dotnet and UWP https://vimeo.com/293302046

ooeygui commented 2 years ago

In Unity 2020, the Native compiler is now used. When the DllImport is present for an API that doesn't exist, the binary doesn't load - it is an untrappable exception.

ooeygui commented 2 years ago

I'll look at it again