rostopic: pluggable message printers

[mikepurvis]

Following on from #724 and related changes, I'd like to explore making rostopic able to scan the workspace and find printer modules able to handle specific message types. There could always be a --raw flag to suppress this behaviour, but in general there are a number of cases where the default message presentation is less than ideal:

• Messages like Log and DiagnosticArray could be presented much better than they presently are.
• Messages with bitfields/enums could show the correct constant name(s) rather than just the number.
• Covariance matrices could be shown in a properly aligned tabular format.
• In general, rostopic becomes smarter without needing a lot of built-in knowledge about specific message types.

In terms of an implementation, it would probably be something that would show up in the package.xml, eg:

<package>
  <name>my_fancy_package</name>
  ...
  <export>
    <rostopic>
      <echo_plugin type="my_fancy_package.RostopicFanciness" />
    </rostopic>
  </export>
</package>

So rostopic (when not in --raw mode) would check the workspace package XML files for such export tags and import the specified classes, passing messages through them for processing as required (probably subject to an optional blacklist).

Would like to get a temperature on whether this interests people. Part of the work would probably involve refactoring this 2000+ line monster into some separate files, and adding some tests to cover core functionality.

[gavanderhoorn]

I like the idea in general, but am worried about performance impact. rostopic should be fast, and especially on resource-restricted systems, crawling the file-system is something that can take ages. I'm not saying that should stop you from doing this (as it definitely seems like a good idea), but it's something to keep in mind when implementing this.

catkin_tools has something similar with the verbs, but afaik that uses setup_tools entry points, correct? rospkg would seem to cache things, so may be more suitable (but would perhaps need a --force-discover like rqt).

[mikepurvis]

Yeah, definitely something to watch out for.

I like the catkin_tools approach, and have some exposure to it (I'm the author of a WIP documentation plugin that Clearpath is already using in production internally). It's possible that something similar would work for catkin-wrapped Python packages, but it will take experimenting to see.

Alternatively, rospack plugins is the canonical way of surfacing this info, and I believe it has some kind of caching going on under the hood: http://docs.ros.org/independent/api/rospkg/html/rospack.html#rospack-plugins-attrib-attrib-package

There's already precedent for Python-based plugins, see:

https://github.com/ros-visualization/rqt_common_plugins/blob/master/rqt_publisher/package.xml https://github.com/ros-visualization/rqt_common_plugins/blob/master/rqt_publisher/plugin.xml

[NikolausDemmel]

I like this a lot.

I'm not sure which plugin scheme would be best suited.

Just as a note, there was some concern about the performance of python entry_points in catkin-tools: https://github.com/catkin/catkin_tools/issues/297

[gavanderhoorn]

@NikolausDemmel wrote:

Just as a note, there was some concern about the performance of python entry_points in catkin-tools: catkin/catkin_tools#297

yes, that was my concern as well. Whatever extension mechanism is used, it should not interfere with the normal (ie: current) usages of rostopic.

[wjwwood]

+1 for the idea in general. One more thing to consider with the plugin lookup and preserving fast start time for rostopic's normal operation, is that rqt uses a ROS_PACKAGE_PATH based lookup mechanism (as @mikepurvis pointed out), but it caches them by default. You have to pass an option to force it to redo plugin "discovery". This keeps the start time pretty quick. Perhaps a similar mechanism could be used here.

[mikepurvis]

A much more intense version of this proposal would be a C++ rewrite based on variant, using the C++ plugin system. However, that introduces a new dependency for a core tool, and I don't think there's really much to be gained in terms of performance.

A logical next step is at least breaking up the existing source into multiple verb-oriented modules.

[spaghetti-]

I'm definitely interested in this as well, particularly in pretty printing covariance matrices, extracting diagonals and of course, displaying bitfields correctly. +1