The MRTK for ROS is being developed in the open: Feature requests are very much welcome at the issues page.
The Robot Operating system - also called ROS [^1] - is an open source middleware for creating Robots. This repository implements glue which connects your Mixed Reality Robotics application with Robots and infrastructure using ROS2. ROS2 runs directly on the Hololens without going through ROS Bridge - reducing latency and improving network efficiency. The ROS2 MRTK extension works with ROS2.net[^2] within the Unity Game engine, allowing you to write ROS2-like code directly in Unity.
The Mixed Reality Toolkit is an Open Source framework for building Virtual Reality and Augmented Reality applications, which target numerous platforms - including Hololens.
This depends on ROS2 Foxy UPM supplied by Microsoft. Windows is required for Hololens and Windows Mixed Reality development. MRTK for ROS2 development will not be supported on other platforms.
reg add HKLM\SYSTEM\CurrentControlSet\Control\FileSystem /v LongPathsEnabled /t REG_DWORD /d 1 /f'
Create a 3D Unity application
Use the Microsoft Mixed Reality Feature Tool, and select the following features:
Wait for Unity to deploy these components.
new Input System
, select Yes
. When Unity restarts, you should now see a window titled MRTK Project Configurator
Unity OpenXR Plugin
Show XR Plug-in Management
OpenXR
.OpenXR
MRTK Project Configurator
Window and select Apply Settings
, then select Next
Microsoft Spatializer
in the Audio Spatializer
dropdown, then click Apply
When Unity Restarts (again), Select the Mixed Reality
Menu item, Toolkit
, Add to Scene and Configure
Add Nuget for unity to your application by double clicking the the NuGetForUnity.3.0.2.unitypackage
. Import the assets into your project.
In the Menu for Unity, select NuGet
, then Manage Packages
.
In the Search edit box, enter Microsoft.MixedReality.QR
, then click search. Install 0.5.3 or later.
In the Window
Menu, select Package Manager
.
Select +
, then Add Package from tarball
. select the Unity release for ROS2 Native downloaded above.
Select +
, then Add Package from tarball
. select the Unity release for the Mixed Reality Toolkit for ROS2 downloaded above.
In the Project
tree view in Unity, expand ROS2 and ROS2.net Native components for Unity
and locate the link.xml
file. Drag and copy this file into your asset folder.
Configure your project for Hololens:
Edit
then Project Settings
Player
. Select the Windows icon.File
Then Build Settings
Add Open Scenes
to add your current scenes to the project.Universal Windows Platform
and Switch Plaform
if needed.Build
New Folder
, name it App
, ensure App
is selected, then click Select Folder
App
folderRelease
build type. (This is important, as the ROS2 binaries are only delivered as release)ARM64
.You can now add scenes from the ROS MRTK extension or use individual components in your application.
One you have configured the application above, you can now add components to your scene to define your application. A Sample scene is provided which demonstrates spatial pinning and Lidar visualization.
ROS2 and ROS2.net UI components for Unity
.Basic ROS World
and select Import.Contributions are welcome. Refer to the Contribution Process for more details.
[^1]: ROS is a trademark of Open Robotics. [^2]: ROS2.net is maintained by the ROS2.net maintainers and community.