search

ros-perception / vision_msgs

Algorithm-agnostic computer vision message types for ROS.
Apache License 2.0
155 stars 74 forks source link

A message for features ? #13

Closed artivis closed 6 years ago

artivis commented 6 years ago

Wouldn't it be interesting to have a ros message to pass features around (think SIFT, SURF, ORB, ED-line etc) ?
Including some helper functions to handle the back and forth conversion with OpenCV.

Kukanani commented 6 years ago

Definitely an interesting idea that we thought about when we were first making this package. So far, we have not heard a compelling reason to add special message types, but I'm happy to discuss further. A few of my thoughts:

  1. Some of these descriptors can be of variable size. For example, there are many variants of HOG with different angle bins, etc. For this reason, it doesn't make sense to me to make a new message type for each descriptor because this would quickly become a huge list of descriptor messages that may be very rarely used.

  2. On the other hand, a single catch-all "visual feature" message would essentially just be an array of floats. Which begs the question, why not use a FloatArray message, which would be more portable? The main point of this package is to be algorithm-agnostic. Messages for each feature type are decidedly not algorithm-agnostic. Putting those features into a generic wrapper doesn't seem to solve the problem, because you still have to use a specific algorithm to understand what the feature means.

Summarizing the above 2 points: I'm open to discussion, but so far I haven't been convinced that making new messages for existing features is useful/in harmony with the overall goals of the package.

Other points

  1. Some of these descriptors may actually be protected by patents (I'm thinking of SIFT here). Whether that patent is on the algorithm or the descriptor itself, I'm not sure. But it would be something I'd want to look into before explicitly incorporating SIFT into the package.

  2. OpenCV conversion functions are where it gets interesting. If there is significant legwork required to convert each specific descriptor type into a ROS-friendly message (i.e., other than just copying a float array), then a set of helper functions sounds useful.. However, I would argue that these belong in a new package to avoid adding an OpenCV dependency to this one!

  3. Many people decide they want to track their own totally custom features (example) which results in custom message types. This is fine, but it's not something that this package would be designed to support.

Kukanani commented 6 years ago

This hasn't seen any activity and I think I've adequately answered the original question, so closing for now. Feel free to re-open if you have additional thoughts.