josephduchesne
commented
6 years ago Hey, I deliberately didn't try to bring up a comparison at the ROSCon lightning talk, since given the short amount of time, I didn't want to come across as badmouthing other community members. Stage and STDR are both reasonable simulators if they fit your needs. Here's my attempt to compare them. Feel free to correct me if I'm mistaken about any of the other simulators.
Here's a breakdown of reasons why you might want Flatland vrs. other options:
STDR vrs. Flatland
- STDR can't do other robots perceived as obstacles from sensors, wheras flatland handles this just fine. This is important
- STDR doesn't do noise in sensors - flatland has gaussian laser noise simulation and noise options in a few other places.
- Kinematic models - Flatland supports doing this, although the current motion models are written in a way that doesn't take full advantage of the physics engine.
- Plugins - All sensors and motion models are written as flatland model plugins using the standard ROS pluginlib. This means that it's really easy to add custom sensors, or plugins that make a model perform specific actions to allow for testing complex scenarios automatically.
- STDR has a nice graphical robot model editor. If you can build your robot using this editor, it can help you get up and going really quickly. Flatland does not have this, and you have to create your robot model using a YAML file, trial and error.
- License - STDR is GPLv3, wheras Flatland is BSD 3 clause. If you want to make proprietary flatland enhancements and distribute the binary, you can under BSD 3 clause. If you want to make flatland enhancements and release them as GPL, you can.
Player / Stage vrs. Flatland:
- Flatland is more tightly integrated with ROS (TFs, visualisation messages, pluginlib etc.) , whereas Player/Stage is more abstracted from ROS. Which design choice is better for you depends on what you're trying to do, but if you're in the ROS environment you may find flatland easier to integrated.
- Flatland plugins are more free-form. Rather than a plugin implement a rigid API, they just attach to events that they're interested in and can modify the world however they want. Player/stage plugins provide specific behaviour classes, which makes them easier to swap in/out for eachother, but if you want to create an entirely different class of plugin, it might be more difficult.
- Flatland is currently single threaded, wheras player/stage is multithreaded. This means that flatland's internal communication stays in sync sequentially under most circumstances, but it doesn't scale as well with multiple CPU cores.
- When we were evaluating Player/Stage in Spring 2017, we found that the ROS interface for player/stage, and the current release version documentation didn't line up very well. This made it really hard to work with.
- I think that Flatland's documentation is a bit easier to follow than Player/Stage's, but it still needs some improvements.
- License: Player/Stage is GPLv2 wheras Flatland is BSD 3 clause. As a developer, in my opinion you have more options with BSD 3 clause (although this could be subject to a lot of debate).
Both vrs. Flatland:
- Flatland is newer and more actively developed than either STDR or Player/Stage . This means it's more likely to find unreported bugs in Flatland, which is a downside, although we'll try to fix them or integrate fixes ASAP.
- Flatland supports world plugins, which can provide world initialisation behaviour. For example, you could write a world plugin that generates a random maze or other world type based on parameters and a seed.
- Flatland supports configurable simulation speed (similar to gazebo, step size * update rate = real time factor), whereas I don't believe this is supported in either STDR or Player/Stage.
- Flatland server is 3k lines of code, and 9k including all the other packages, STDR is 15.6k lines of C/C++, Stage is 16k. This is due to Flatland's plugin focus. I believe that this should make it easier to enhance the core, and extend functionality without the server becoming too complex. Complex functionality can be pushed out to plugins via the pluginlib interface. (All line counts based on cloc "code" column for C/C++ and headers).
- Both player/stage and STDR have a sonar simulation, wheras Flatland doesn't have a plugin for that yet. It wouldn't be hard to add (as it's basically a simpler version of the laser scaner plugin), but if you need this ASAP and want Sonar it might be easier to use STDR if it meets your other needs.
- Flatland doesn't have a GUI, and instead uses visualisation messages + rviz plugins to integrate directly into RViz if you need to spawn/move models, or
- Flatland simulates the static world as a series of layers, and all dynamic objects as models. This means that a table and a robot are the same type of object, aside from the robot likely having a bunch of model plugins attached to give it a motion model, sensors, tfs published etc. This design choice should allow for complex functionality to be simulated fairly easily. In the Flatland Turtlebot integration's example world, this can be seen simulating a sliding door that listens to a boolean ros topic, and a passive collision sensor publishing a boolean ros topic working together to simulate a proximity activated door. This could probably also be done in Player/Stage but it would likely involve a lot of messing around and misusing their robot interfaces.
Flatland vrs. Gazebo:
- Under almost all circumstances Flatland should run significantly faster than Gazebo
- In my non-scientific tests, physics in Flatland is more stable than in Gazebo at faster simulation speeds (likely due to less degrees of freedom)
- The physics simulation is not abstracted in Flatland like it is in Gazebo. Pro: It's easier to interact with the physics simulation. Con: You can't swap out physics backends (although realistically there are only 2-3 viable options and we believe we picked the best one).
- Performance in Gazebo degrades as the world size and complexity increase. This also affects Flatland, but not to the same degree. We have no trouble simulating entire shopping malls in flatland. Performance in a very large space (entire large shopping mall) is roughly half of performance in a small space (single room) for our use case.
Hopefully that illuminates some of the differences between your options. There are certainly areas where each of the simulators shine, but the reasons I highlighted here are the primary ones that caused us at Avidbots to spend the time writing our own.
We have the need to simulate large numbers of complex, dynamic environments, and the simulation speed in Gazebo (which we used, and still use) proved a major road block. What we really wanted was a fast, lightweight, 2.5D version of Gazebo that could allow us to easily simulate people walking through areas as the robot was in operation, doors opening and closing, multiple robots interacting with each-other, dynamic environments that change frequently, very large worlds (since our robots clean malls, airports, and other similar large spaces around the world).
If you have any other questions, or are able to explain your use-case in more detail I'd be happy to help.
naveedhd
I am currently considering picking up one 2 (2.5D) simulator and would like to ask what specific use case led you to build this package as opposed to using existing solutions such as stage or stdr?
Watched the ROSCon ligthning talk but missed this comparison.