Soccer msgs containing covariance

[jaagut]

Hey! With this issue, I want to start a discussion about some sort of msgs containing a covariance of an object.

How do we implement this?

• Do we extend the existing 3D msgs? Semantically not great...
• Do we create a new package? Something like soccer_world_model_msgs?
• Covariance (and other world model information) is relevant for which objects? Can we abstract this?
• Which additional information is also relevant?

What are your thoughts? @ijnek @Flova @SammyRamone

[ijnek]

Thanks for bringing this up! I had some thoughts about this before, but failed to decide on something due to the reasons you've raised.

Before we discuss the options, I think we should be on the same page about which covariane we're talking about. Is the covariance you're talking about measurement errors (ie. mixture of imu error, tf error, etc.), or are they covariance of objects after being filtered through a filter (ie. kalman filter or an approximation of particle spread from a particle filter)?

[jaagut]

I was originally thinking about the covariance of objects after being filtered through a filter, as I was working on a Kalman filter. Presumably, both variants are important, but I'm no expert and don't know how/why to separate these.

[ijnek]

This is how I currently imagine the flow of msgs.

covariance of object after being filtered through a filter corresponds to the soccer_model_msgs, which isn't a package formalised yet.

The reason why soccer_vision_3d_msgs and soccer_model_msgs is separated, is because the model can contain more information than what was detected from vision. For example, if you're using a kalman filter, you could get a velocity estimate of the ball.

I have a draft PR in my fork where I was writing up an example soccer_model_msgs package. There, I have

soccer_model_msgs/msg/Ball.msg:

std_msgs/Header header
geometry_msgs/PoseWithCovariance pose
geometry_msgs/TwistWithCovariance twist

# The assumed radius of the ball (in mm)
float32 radius

which stores the pose and twist (velocity) of the ball with covariance, does this seem to fit your needs?

[jaagut]

After a small peek into your fork, it looks promising. Where would we include a covariance to quantify measurement errors?

[Flova]

This is handled by the filter and included in the covariance of the filter state. E.g. uncertain vision measurements should lead to a less accurate world state which is represented by the covariance matrix. The filter state has no direct assosiation to the filter inputs. We previously had a message with covariance aswell as the original detection confidence. This was primarily for situations like the soccer_3d_vision_msgs where a covariance can be used to model the ipm projection errors and still keep the image space information. This was never implemented in the ipm but the msgs had that functionality.

[ijnek]

E.g. uncertain vision measurements should lead to a less accurate world state which is represented by the covariance matrix

Adding to this, the "uncertain vision measurements" aren't related to the "confidence" of the vision detections, they are positional uncertainty (covariance) that come mainly from IMU/ipm errors. I remember I was confused about the two when I was working on localization.

Where would we include a covariance to quantify measurement errors?

I'm feeling that the best option would be to change soccer_vision_3d_msgs/msg/Ball.msg to:

# Describes a detected ball from vision modules, in 3d coordinates.

# Center point of the ball (in meters)
- geometry_msgs/Point center
+ soccer_geometry_msgs/PointWithCovariance center

# Confidence that the detected object is a ball
soccer_vision_attribute_msgs/Confidence confidence

where soccer_geometry_msgs/msg/PointWithCovariance:

# This represents a point in free space with uncertainty.
geometry_msgs/Point point

# Row-major representation of the 3x3 covariance matrix
# The orientation parameters use a fixed-axis representation.
# In order, the parameters are:
# (x, y, z)
float64[9] covariance

and ensure the covariance is estimated and not left empty.

I can proceed with a PR if we agree.

[ijnek]

I've raised a Draft PR for soccer_model_msgs: https://github.com/ros-sports/soccer_interfaces/pull/43

[ijnek]

@jaagut

Assuming you're working on localization or ball tracking currently, I'd be interested in discussing what topics you planning on subscribing and publishing to. I think having the topics not bit-bots specific, would help other teams replace the localization component easily with their own node if they want. What's your plans?

[jaagut]

I was working with others on the ball filter. This ball filter is subscribing to the reprojected balls published by the soccer_ipm module (https://github.com/ros-sports/ipm/pull/7). The ball filter uses a Kalman filter and then publishes one filtered ball using a humanoid_league_msgs/PoseWithCertaintyStamped and a velocity using a geometry_msgs/TwistWithCovarianceStamped.

The humanoid_league_msgs/PoseWithCertaintyStamped msg will be replaced, once #43 gets merged. We will then use the soccer_model_msgs/Ball msg.

[ijnek]

Got it, that seems like something easily replacable by other teams. Sounds good to me.

[jaagut]

Resolved by #43