Camera protocol - update with info about camera addressing and missions

[hamishwillee]

Work related to https://github.com/mavlink/mavlink/pull/2024

This documents the presence of the id parameter in the image command. I've tried to make it clear how addressing works both before and after, and in the command protocol and missions.

@julianoes @rmackay9 @Davidsastresas A couple of things popped up:

1. MAVLink cameras send a HEARTBEAT with the camera type, which tells GCS that it should query for CAMERA_INFORMATION using MAV_CMD_REQUEST_MESSAGE. However an autopilot or onboard computer has its own component ID and type so what should trigger GCS to know it should query the onboard computer?

   The obvious answer is that a GCS could choose NOT to wait for heartbeats, but instead fire off MAV_CMD_REQUEST_MESSAGE for camera information to the system ID with component id = 0. Then all components that support cameras should fire back a response. I think this makes sense except:

   • Do we have to worry about multiple ACK?
   • Do we suggest no longer querying cameras individually.

   Alternatively we suggest that a GCS must also query the onboard computer and autopilot. This has the benefit that it is the same query process as currently. However it does mean that if some other component ends up managing a set of cameras we won't know to address it unless it outputs the heartbeat.

   Thoughts?

2. Each proxied camera has a "camera id". So if I want to send a command to the non-mavlink camera attached to a companion computer, then I send the command to the companion with target address of the companion, and the camera's specific camera id. Similarly, if I want to send the command to a camera attached to the autopilot I an address it.

   However I think we missed the case of a mission command for a camera attached to an onboard computer.

   We can specify the id in a mission of 1-6 but this will be executed by the autopilot on ITS connected cameras. There is no way to say in a mission that you want the command to go to the onboard computer.

   Thoughts? Solutions?

3. How can a system tell if the id is not supported? Do they need to be able to?

[Davidsastresas]

• About 1, I like the approach of leaving up to the GCS to ask or not for the messages with compid 0, or alternatively select manually in GCS settings if needed the component ids you want to ask for it. I don't think multiple ack would be an issue as they should be properly identified by the component id right? For the rest of it, I think this choice is what makes most sense.

• About 2, I think that could be addressed from the autopilot firmware. If the autopilot knows there is a mavlink camera connected to it, even if it is through a companion computer, we should have everything needed to forward the command to the companion computer. So in the meantime, I think it is fine as it is, at the protocol level.

• About 3, I think the current behaviour of an autopilot or component not knowing about the new id field is fine. All connected cameras will be triggered, which is probably the same behaviour they had before. So I don't think we should handle specially this situation

[hamishwillee]

@Davidsastresas FWIW re 2, the issue can't be addressed by the autopilot, because while it can potentially know that a companion computer has particular connected non-MAVLink cameras there is no way to know in a mission which one you want to send the command to.

Fortunately Julian made a suggestion which fixes this and 1. Agree with you on 3. Will post a summary next.

[hamishwillee]

Discussed this in 20240814-Dev-Meeting.

What we're going to do is make it very clear that only autopilots are allowed to proxy components. A companion computer or other component (except autopilot) that wants to support multiple hardware-connected cameras must emulate it as a MAVLink camera.

• This fixes camera discovery ("1"), because essentially we no longer have arbitrary unknown components that might support cameras - we only have those that identify themselves as cameras and the autopilot.
  • So now the process is to request CAMERA_INFORMATION from all thngs that identify as cameras AND the autopilot.
  • A ground station might also broadcast the request - not sure what is better but will not suggest this in the docs unless there is strong support.
• This fixes addressing all cameras in missions (2) and commands - because if you're targeting a camera attached to a companion it has a MAVLink identity, so you can use its component ID.

Note that the only reason we allow flight stacks to proxy is because they have prior art of connected cameras and they are "unable" to emulate mavlink cameras for them. There is no reason to extend this to companions.

FYI @julianoes says he's going to look into MAVSDK allowing support for multiple camera components on the same companion, so this will not necessarily even be onerous.

I'll update the docs next.

[hamishwillee]

@julianoes OK, I have updated to match https://github.com/mavlink/mavlink-devguide/pull/517#issuecomment-2290360512

Can you please sanity check?

[hamishwillee]

I believe that we now have a clear picture on camera discovery and addressing that also covers device-attached autopilots.

Nothing has changed for MAVLink cameras except that now specific cameras can be targeted in missions by specifying their component ID in the respective command's "Target camera id" parameter. Discovery and identity is done in exactly the same way.

We now allow autopilot-attached cameras. The origin of messages from these is identified by the value 1-6 in message camera_device_id field (0 for mavlink cameras). THese can be targeted in commands and missions using the id in the "Target camera id" parameter.

There is one change for discovery that is a GCS should query all autopilots for camera_information.

Further (just added), the command_ack should include the camera id of an attached camera in the result_param2. That would just be for information.

@julianoes et al, I'm going to merge this. I am happy it matches other discussions on this topic, and I am separated from it enough now to effectively subedit/review.