search

robotics-in-concert / rocon_multimaster

Key components for ros multimaster systems
12 stars 19 forks source link

Multirobot concert on kinetic with Unity as simulator #349

Open samiamlabs opened 5 years ago

samiamlabs commented 5 years ago

Hi,

I worked on a project last year where I used parts of the indigo concert system to make a miniature forklift solution http://minireach.readthedocs.io/en/latest/demo.html

image

Just started working on an opensource project where I plan to implement something similar with a mini forklift made out of turtlebot3 parts. https://dyno-docs.readthedocs.io/en/latest/simulation/route_scheduler.html

Have implemented some basic Robot Apps, like I simple waypoint scheduler using py_trees and a simple react/material-ui web interface.

Ran into a performance and stability bottleneck with Gazebo and the standard plugins in the last project. Thought I would try to use Unity as a multi-robot simulator this time. (I have various other platforms and projects I want to use with the same framework as well)

https://www.youtube.com/watch?v=7NCSMg_bq1I image

Am currently trying to get multiple robots/forklifts working in Unity using rocon (or at least rocon_gateways) and have some questions:

// Samuel

stonier commented 5 years ago

That's awesome. What's the connection with Toyota?

Before I get into the details, some context. After indigo, we did a significant upgrade to multimaster, the rapp manager, interactions and the remocon to both simplify and robustify it on the gopher fleet we were building and testing on. This is what you see on 0.8-indigo-gopher, 0.8-kinetic and devel branches. The highlights:

If you're upgrading, that means quite a few changes, but hopefully a vastly improved set of tools that have been hardened (consider indigo the proof of concept). The multimaster framework is effectively the same, rapps, interactions and the remocon are where you'll have the largest delta. I don't think you were using the concert side software (scheduler, etc) so probably not so much to worry about there.

Unfortunately, I never got to overhaul the documentation and tutorials like I did to py_trees for kinetic. If you're already familiar with everything though, it shouldn't be too hard to walk through with what you'll find (demo launchers, etc).

I noticed that the version of rocon_multimaster in the current deb release for kinetic is missing the redis-3.0.conf files (because you removed them). This means that the hub shuts down with an error message (rosdep installed redis-server 3.0 on kinetic).

They didn't get taken out, looks like it's merely because the redis-3.0.conf files in the devel branch didn't get released yet. Try the devel branch and if it works, send a PR?

The lines changed here: https://github.com/robotics-in-concert/rocon_multimaster/commit/42bfedc71bbb15bf150b82738b8d88ea3e9f1030 throws an AttributeError on my system and prevents gateway_info from being published (and lots of stuff stops working). Commenting them out fixed the problems I was having, but I don't know the cause.

That'll need some more digging - looks like something going wrong in the construction of your hub discovery thread which has left those variables uncreated. Do you have a simple, reproduceable example?

You are referencing a "gopher style concert" here: https://github.com/robotics-in-concert/rocon_app_platform/blob/devel/rocon_app_manager/launch/concert_client.launch Is there any documentation for how to configure one of those? How does it differ from that described in http://wiki.ros.org/rocon/indigo/Guide ?

See above, none. If the separate bits and pieces start working, it should be self explanatory if you're familiar with the old one though. There are demo launchers in rocon_interactions/launch and rocon_app_manager/launch (trace them back to configuration files for everything) and the old tutorials should still be mostly applicable for the gateways.

samiamlabs commented 5 years ago

What's the connection with Toyota?

I ran the minireach project at the "concept evaluation"/research department of https://toyota-forklifts.eu/ They are a part of the Toyota Industries Corporation and manufacture electric forklifts. There is very little collaboration between this company and Toyota Motors (projects like the Partner Robot Family) or any of the Japanese research organizations under the Toyota umbrella.

I was evaluating if a ROS based system could be used to replace the expensive and oldfashioned AVG control systems from Kollmorgen they are currently using for automatic forklifts (https://ndcsolutions.com/).

The proof of concept turned out reasonably well, but it did not seem like the organization would be able to bring a system like this to market. There are pretty much only mechanical engineers in the technical staff and management are not used to software projects at all. Was pretty clear that they were going to continue outsourcing control and fleet management systems for this reason so I decided against renewing my contract when the project was finished. Am currently trying to get a service robot startup off the ground with some people I know from college http://dynorobotics.se/

They didn't get taken out, looks like it's merely because the redis-3.0.conf files in the devel branch didn't get released yet. Try the devel branch and if it works, send a PR?

The redis-3.0.conf on the devel branch seems to work, but the hub_discovery issues prevent pretty much anything else from working on that branch on my system. Removed a couple of lines of code in https://github.com/samiamlabs/rocon_multimaster/tree/kinetic-fixes which made it usable. Did not actually fix the problem though, so didn't think it was appropriate to make a PR. You should be able to reproduce the problem with any of the gateway or concert tutorials. I'm using a static IP for ROS_MASTER_URI if that is relevant.

I have multiple trucks running gopher rapp managers working with Unity now: https://youtu.be/EkMigTqas2U Have some issues with the rosbrige based WebSocket interface getting overloaded with too much laser_scan data, but I'm planning to move to https://github.com/ros2/ros1_bridge and https://github.com/firesurfer/rclcs at some point. Hopefully, they can handle the traffic better.

Had a lot of problems with interactions nondeterministicly crashing when launched in the minireach project, so I was not planning to use that system this time. Is it more stable in the gopher releases?

Have been talking to a group of undergraduates at Korea Aerospace Univ that wanted to use the minireach project as a base for a drone delivery system they are working on based on http://kcl-planning.github.io/ROSPlan/ Had to tell them no because my old boss from Toyota did not let me release that project as open source when it was done (despite an informal promise from my previous supervisor that quit before me).

Thought that I would attempt to integrate rocon_multimaster with rapp manages and py_tree based task execution with ROSPlan for planning/dispatch on the "concert" side to help them out in this new project. I don't really know if that will work yet as I'm not familiar with ROSPlan.

I think I will try to make a new world-state system analogous to world_canvas using https://pouchdb.com/ and eventually attempt to integrate something similar to http://wiki.ros.org/visp with cartographer submaps and https://wg-perception.github.io/object_recognition_core/ for object detection.

Have done some work extending ork to support training yolov3 with data from rendered 3d-meshed and images from ordinary training datasets like VOC2012 as background to bootstrap detection. It looks pretty promising so far but is not very user-friendly at the moment...

// Samuel