search

fsstudio-team / ZeroSimROSUnity

Robotic simulation in Unity with ROS integration.
https://roboticsimulationservices.com/
MIT License
176 stars 21 forks source link
2d-lidar 3d gazebo image-segmentation laserscan lidar machine-learning mobile-robot-navigation mobile-robots reinforcement-learning reinforcement-learning-environments robot-operating-system robotic-arms robotics ros sensors simulation unity

readme

ZeroSim

ZeroSim is a robotics simulation engine built on the easy to use Unity 3D development platform and the power of the Robotics Operating System (ROS). ZeroSim is designed for ease of use and rapid development of all sorts of robotics and simulation -- from warehouses and industrial settings, to farming and outdoors -- from robotic arms to ground and drone based mobile robots.

ZeroSim is a project developed over several years by FS Studios for the rapid development of all sorts of robotic simulation projects for various clients, from robotic arms to mobile robots.

We are releasing ZeroSim as open source to support the community of roboticist and software engineers that have supported us over the decades. We are in active development and welcome all feature requests, bug reports, and pull requests.

MoveIt Example Mobile Robot Example

Overview

ZeroSim provides a multitude of tools for building robots and environments in Unity to interface with ROS. We strive to provide the same functionality and ROS interfaces of Gazebo. Including:

API Documentation

API Documentation

Getting Started

Video Tutorials

Tutorial #1: Install ZeroSim in Unity

Tutorial #2: Create a New ZeroSim Unity Scene

Tutorial #3: Connect to ROS

Tutorial #4: Build a Robot From Scratch

Tutorial #5: Import URDF Robot Universal Robots UR10 Arm

https://youtu.be/VJyvjJUfjv8

Recommended System

Setting up a new Unity Project

  1. In Unity Hub create a new Unity project using Unity 2020.x or later. New Unity Project
  2. Add ZeroSim via Unity Packages:
    1. IMPORTANT: Make sure GIT LFS is installed on your system before continuing!
    2. Unity Menu Window -> Package Manager
    3. Select the + dropdown:
      Dropdown
    4. Select Add Package From Git URL... and enter git@github.com:fsstudio-team/ZeroSimROSUnity.git. Note this can take upto a few minutes to update but you should see the following:
      ZeroSim Package Installed
    5. Import the ZeroSim Samples by selecting the Samples Import button in the Package Manager:
      Import ZeroSim Samples
  3. IMPORTANT the default Unity physics settings do not work well with a lot of simulations. It is very much recommended (required for probably most all simulations) to set the physics settings by opening the Unity menu Edit -> Project Settings... -> Physics and set the Default Solver Iterations to 30 and the Default Solver Velocity Iterations to 60. Unity Physics Settings
  4. IMPORTANT the default Unity fixed timestep setting does not work well with a lot of simulions. It is very much recommended to set the Fixed Timestep setting by Edit -> Project Settings... -> Time to 0.005 Timestep Settings
  5. If running Unity on Linux you want to avoid using OpenGL and use Vulkan, otherwise image based sensors may run slowly or not at all. To change to using Vulkan:
    1. In the Unity Menu: Edit -> Project Settings...:
    2. Uncheck Auto Graphics API for Linux and then under Graphics APIs for Linux set Vulkan ahead of OpenGL:
      Vulkan Settings

Getting ZeroSim ROS Docker Container

  1. Available at https://hub.docker.com/r/zerodog/zerosim_ros_vnc or docker pull zerodog/zerosim_ros_vnc:latest

    Running TurtleBot Test Scene

NOTE: Order of operations is important. Especially starting the Docker before the Unity simulation.

  1. Make sure that the ZeroSim samples are installed as outlined above.
  2. Make sure that the ZeroSim Docker container above is installed.
  3. Open the Turtlebot3_Waffle_test.scene Open Turtlebot Scene
  4. Launch the ZeroSim Docker via: 
    
docker run -it --rm \
--publish=9090:9090 \
--publish=11311:11311 \
--publish=8083:8083 \
--publish=80:80 \
--publish=5678:5678 \
--name my_zerosim_vnc_docker \
zerodog/zerosim_ros_vnc:latest \
roslaunch zero_sim_ros basic_unity_editor.launch
5. Run the ROS teleop in a seperate terminal by running:

docker exec -it my_zerosim_vnc_docker \ bash -c "source devel/setup.bash ; rosrun turtlebot3_teleop turtlebot3_teleop_key"

6. In the Unity editor press the "Play" button.
7. The Turtlebot can now be controlled via the `w a s d` keys in the ROS teleop console window:

### Using RViz for Turtlebot

This will show visualizing the 2D LIDAR and ROS TF in RViz.  This uses a VNC viewer to the ZeroSim ROS Docker container.

1. Startup the Turtlebot Test Scene as detailed above.
2. Open a noVNC connection by:
   1. In a browser open http://localhost:8083/vnc.html
   2. Press the "Connect" button. ![noVNC Login](Documentation~/images/novnc_login.png)
3. In the VNC window press the *LEFT* mouse button and select "Terminal". ![noVNC Terminal](Documentation~/images/novnc_terminal.png)
4. In the new terminal run `rviz -d ./src/zero_sim_ros/rviz/turtlebot_viewer.rviz`.  RViz will start up with a 3D view with the LIDAR scanner visibile. ![RViz Turtlebot Viewer](Documentation~/images/rviz_turtlebot.gif)

### Running Universal Robot UR10 Arm Test Scene with MoveIt!

*NOTE:* Order of operations is important.  Especially starting the Docker *before* the Unity simulation.

1. Make sure that the ZeroSim samples are installed as outlined above.
2. Make sure that the ZeroSim Docker container above is installed.
3. Open the `UniversalRobot_UR10_test.scene` ![Open UR10 Scene](Documentation~/images/ur10_test_scene.png)
4. Launch the ZeroSim Docker via:

docker run -it --rm \ --publish=9090:9090 \ --publish=11311:11311 \ --publish=8083:8083 \ --publish=80:80 \ --publish=5678:5678 \ --name my_zerosim_vnc_docker \ zerodog/zerosim_ros_vnc:latest \ roslaunch zero_sim_ros ur10_moveit.launch

5. Start the UR10 test Scene by pressing the Play button.
6. Open a noVNC connection by:
   1. In a browser open http://localhost:8083/vnc.html
   2. Press the "Connect" button. ![noVNC Login](Documentation~/images/novnc_login.png)
7. In the noVNC window RViz will be setup with MoveIt ![UR10 RViz MoveIt](Documentation~/images/ur10_moveit_rviz.png)
8. You can now to standard MoveIt! operations in RViz to control the UR10 arm ![UR10 MoveIt](Documentation~/images/ur10_moveit_rviz.gif)

### Running Image Segmentation Test

1. Make sure that the ZeroSim samples are installed as outlined above.
2. Make sure that the ZeroSim Docker container above is installed.
3. Open the `ImageSegmentation_test.scene` ![Open Segmentation Test Scene](Documentation~/images/image_segmentation_scene.png)
4. Launch the ZeroSim Docker via:

docker run -it --rm \ --publish=9090:9090 \ --publish=11311:11311 \ --publish=8083:8083 \ --publish=80:80 \ --publish=5678:5678 \ --name my_zerosim_vnc_docker \ zerodog/zerosim_ros_vnc:latest \ roslaunch zero_sim_ros basic_unity_editor.launch

5. Start the Image Segmentation Test Scene by pressing the Play button.
6. Open a noVNC connection by:
   1. In a browser open http://localhost:8083/vnc.html
   2. Press the "Connect" button. ![noVNC Login](Documentation~/images/novnc_login.png)
7. In the VNC window press the *LEFT* mouse button and select "Terminal". ![noVNC Terminal](Documentation~/images/novnc_terminal.png)
8. In the new terminal run `rqt_image_view /image/segmentation_image`.  
9. Open up a second terminal and run `rqt_image_view /image/image_raw` ![RQT Image View Segmentation](Documentation~/images/rqt_image_view_segmentation.png)

### Export URDF

1. Make sure that the ZeroSim samples are installed as outlined above.
2. Open scene the `URDF_test.scene` in the ZeroSim samples.
3. Select `SimpleRobotArm` in the scene hierarchy.   
![Select Simple Robot Arm](Documentation~/images/select_simple_robot_arm.png)
4. Select `Export URDF` in the root properties view. ![Export URDF](Documentation~/images/export_urdf.png)
5. Select the directory to export to.
6. An excellent online URDF viewer is available: https://gkjohnson.github.io/urdf-loaders/javascript/example/index.html  Just drag and drop the files exported above.

### Import URDF

1. Right click and select `ZeroSim --> Import URDF...`

#### LEO Robot Example

1. Clone the Leo robot: `git clone --recursive https://github.com/LeoRover/leo_common.git`
2. Checkout Melodic version.  `cd leo_common && git checkout melodic`.
3. Startup ZeroSim Docker mounting the LEO directory:

make sure you are in parent directory for leo_common

cd ..

run docker

docker run -it --rm \ --publish=9090:9090 \ --publish=11311:11311 \ --publish=8083:8083 \ --publish=80:80 \ --publish=5678:5678 \ --name my_zerosim_vnc_docker \ --volume=$(pwd)/leo_common/:/catkin_ws/src/leo_common/ \ zerodog/zerosim_ros_vnc:latest \ bash

4. In the Docker command prompt build the catkin workspace:

source devel/setup.bash catkin build

Make sure to source ROS again to get the new LEO robot

source devel/setup.bash

5. In the Docker command prompt Run XAcro on the LEO robot to get the URDF:

rosrun xacro xacro src/leo_common/leo_description/urdf/leo_sim.urdf.xacro > /tmp/leo_sim.urdf

6. Convert LEO robot meshes to .obj for Unity.  In the Docker command prompt:

/zerosim_tools/convert_meshes_to_obj.sh ./src/leo_common/leo_description/models


7. Copy URDF from Docker to host:

create a directory and a sub-directory to store the URDF and meshes

mkdir -p my_leo_robot/leo_description

copy the URDF

docker cp my_zerosim_vnc_docker:/tmp/leo_sim.urdf ./my_leo_robot 


8. Copy Meshes from Docker to host:

Note we are preserving the path

docker cp my_zerosim_vnc_docker:/catkin_ws/src/leo_common/leo_description/models ./my_leo_robot/leo_description


9. Fix the URDF paths to the meshes:

First fix up the .DAEs to point to the OBJs

sed -i 's#.dae#.obj#g' my_leo_robot/leo_sim.urdf

The fix up the .STLs to point to the OBJs

sed -i 's#.stl#.obj#g' my_leo_robot/leo_sim.urdf

Now remove the package:// because we are using the filesystem

sed -i 's#package://#./#g' my_leo_robot/leo_sim.urdf 



10. In Unity import URDF by: Right click and select `ZeroSim --> Import URDF...`