Gimbal enhancement list

[rmackay9]

This is a consolidated list of gimbal enhancements:

• [ ] https://github.com/ArduPilot/ardupilot/issues/26892
• [ ] https://github.com/ArduPilot/ardupilot/issues/26004
• [ ] https://github.com/ArduPilot/ardupilot/issues/24571
• [ ] https://github.com/ArduPilot/ardupilot/issues/24093
• [ ] https://github.com/ArduPilot/ardupilot/issues/24013
• [x] log gimbal's reported angles and rates if available
• [ ] MAVLink gimbals (e.g. Gremsy) should log gimbal information (vendor, model, etc) to the logs. currently this info is only in a send_text which is displayed to the user but won't appear in the logs because it is normally sent before the vehicle is armed
• [ ] MAVLink gimbals (e.g. Gremsy) should limit display of vendor and model to 32 characters (e.g. replace %u with %u.32).
• [ ] https://github.com/ArduPilot/ardupilot/issues/22148
• [ ] https://github.com/ArduPilot/ardupilot/issues/23149
• [ ] https://github.com/ArduPilot/ardupilot/issues/21665
• [ ] https://github.com/ArduPilot/ardupilot/issues/22900
• [ ] https://github.com/ArduPilot/ardupilot/issues/22440 (not sure if this is worth it)
• [ ] https://github.com/ArduPilot/ardupilot/pull/23092
• [ ] consume GIMBAL_MANAGER_SET_MANUAL_CONTROL messages
• [ ] enforce mavlink control only by sysid/compid received from MAV_CMD_DO_GIMBAL_MANAGER_CONFIGURE message

The corresponding Camera enhancement list is here https://github.com/ArduPilot/ardupilot/issues/23151 The MavlinkV2 gimbal spec is here

Completed items

• [x] https://github.com/ArduPilot/ardupilot/issues/7658
• [x] AP_Follow support for mounting following lead vehicle (PR)
• [x] support sending GIMBAL_MANAGER_STATUS (perhaps only if message-interval has been used to request it be sent) https://github.com/ArduPilot/ardupilot/pull/23737
• [x] https://github.com/ArduPilot/ardupilot/issues/23042
• [x] respond to requests for GIMBAL_MANAGER_INFORMATION (see https://github.com/ArduPilot/ardupilot/pull/23587)
• [x] consume GIMBAL_MANAGER_SET_ATTITUDE messages
• [x] health reporting and pre-arm check
• [x] increase rate control's maximum speed (see MNT_JSTICK_SPD parameter)
• [x] extend 60s timeout for discovering a gimbal. Perhaps allow the gimbal to be discovered as long as the vehicle is disarmed
• [x] DO_GIMBAL_MANAGER_PITCHYAW mission and immediate command should support
  • [x] both earth-frame vs body-frame angles (currently always earth-frame)
  • [x] earth-frame and body-frame rates (how can we do this? users would need to add NaN to angles?)
• [x] add GIMBAL_MANAGER_SET_PITCHYAW message support (not to be confused with MAV_CMD_DO_GIMBAL_MANAGER_PITCHYAW) -- cancelled
• [x] bring other gimbal drivers up to the same level as the Gremsy driver including
  • [x] body-frame and earth-frame control during missions
  • [x] pilot control of lock vs follow (during angle or rate control)
  • [x] consume attitude information from gimbal
  • [x] health reporting
  • [x] send gimbal-device-attitude-status messages (PR)
• [x] Lua script support
  • [x] set angle and rate (both body-frame and earth-frame)
  • [x] set target position (aka ROI)
  • [x] set gimbal mode (relaxed, ROI, mavlink targetting, etc)

These ground stations Issues and PRs should also be resolved/merged

• [x] MissionPlanner PR : mavcmd: add DO_GIMBAL_MANAGER_PITCHYAW command
• [ ] MissionPlanner Issue : Plan: ATTITUDE_TIME climb rate field reset to zero
• [x] Mission Planner PR : MAVLinkInterface: scale DO_GIMBAL_MANAGER_PITCHYAW
• [x] MAVProxy Issue : WP Editor corrupts command's Param5 if not Latitude

[olliw42]

fantastic list

• on the very top of the list I would put removing the need for the private mavlink channel thing (I'm not sure that's still required, but the discourse thread makes me believe so)
• I also would put on high priority sending the proper estimator flags in AUTO_PILOT_STATE_FOR_GIMBAL_DEVICE. Currently you simply send your ArduPilot specific EKF flags, which is really kind of rude, and outside of any standard.
• generally: get the communication in line with gimbal v2
• especially: removal of any old MOUNT_xxx stuff in gimbal v2 based gimbal drivers (like gremsy), including the modes stuff
• concerning gimbal discovery time: I indeed do it that way in BetaCopter/Plane, to link it to armed/not armed
• support also in other vehicles, especially plane
• sending AUTO_PILOT_STATE_FOR_GIMBAL_DEVICE only to the gimbal, and not system-wide broadcast
• support of landing state flag in AUTO_PILOT_STATE_FOR_GIMBAL_DEVICE, for all vehicles, also plane

ground station/MP issues

• support of GIMBAL_DEVICE_ATTITUDE_STATUS, i.e. fill campointa/b/c from this
• support of GIMBAL_MANAGER_INFORMATION, if it finds a gimbal heartbeat it should request it, and display the info in some useful location
• make it behave nicely with mavlink components, e.g. when they do not have parameters, or do not support other things a fc appears to be implicitly expected to support, like AUTOPILOT_VERSION, etc (see also https://github.com/ArduPilot/MissionPlanner/issues/2672).

:)

[rmackay9]

@olliw42,

Txs very much for the list. By the way, I've created another PR (see link above) which I hope takes us one more step forward. Also I think I need hardware to test all our various drivers so I've raised a proposal including for a SToRM32 gimbal. Any feedback or suggestions are greatly appreciated.

[julianoes]

@rmackay9 good list, and great effort adding support!

If I could add one item to the list, it would be to change the name of the driver to "MAVLink Gimbal v2" rather than Gremsy as the goal would be to have this as a generic protocol rather than a Gremsy specific one. Of course it can still be noted in the description that it works with Gremsy.

If you have questions or gripes with the gimbal v2 docs, ping me, and I'll try to help.

[olliw42]

@julianoes

If I could add one item to the list, it would be to change the name of the driver to "MAVLink Gimbal v2" rather than Gremsy as the goal would be to have this as a generic protocol rather than a Gremsy specific one

there had been some discussion on this here: https://discuss.ardupilot.org/t/gimbal-hardware-for-randym/87220/5

Let me cite the relevant part from Randy's post: "I certainly like the idea of a single MAVLink gimbal driver but so far I’ve seen 3 different mavlink implementations (STorM32, Gremsy, NextVision) and they’re all different enough that I fear I would be unable to actually make it work. So for now I think I’m going to stick with separate drivers and hopefully the day will come where we can make them all inherit from the same base class."

[rmackay9]

@julianoes,

Yes, what @olliw42 says is the current state. I'm not opposed to it, I just need at least one more example gimbal that can use the driver as-is. Instead of renaming the current one, I would probably create a new enum entry (maybe called "MAVLink Gimbal v2") that uses the same driver under the cover but let's see what happens...

Txs very much for the offer re the gimbalv2 docs.

We still have a couple more messages to add support for and then I was thinking of removing the WIP tags on at least the messages that we're using in AP.

[rmackay9]

@julianoes,

I've realised that the new mavlink messages don't have the concept of MAV_MOUNT_MODE. At the moment, we're continuing to support the DO_MOUNT_CONTROL and DO_MOUNT_CONFIGURE commands that allow setting the mode but I wonder how you were thinking we could do without the mode concept and the ability to set it?

In particular I think the pilot (or mission or lua script) needs to be able to specify the gimbal go into special modes like the Retracted position (which I'm interpreting as the gimbal relaxing the motors) or Neutral position (which is normally pointing forward). These two seem to be handled using the the GIMBAL_MANAGER_FLAGS which has neutral and retract bits.

.. but the pilot also wants to be able to specify that they're in control using the RC transmitter which they do by setting the mode to RC_TARGETING. I suspect you will say that we should use MAV_CMD_DO_GIMBAL_MANAGER_CONFIGURE but "RC input" is not a mavlink system so it doesn't have a sysid/comp-id.

.. there are other gimbal modes that we support including pointing at home, pointing at a sysid (e.g. point at another vehicle), etc.

[julianoes]

These two seem to be handled using the the GIMBAL_MANAGER_FLAGS which has neutral and retract bits.

Yes, that was the idea there.

but "RC input" is not a mavlink system so it doesn't have a sysid/comp-id.

So to signal that RC is in control, I use the sysid_primary_control and compid_primary_control of the autopilot (for PX4 that's by default 1, 1) which means that itself is in control, and itself has an RC receiver. I understand this is a bit of a kludge...

pointing at home, pointing at a sysid (e.g. point at another vehicle)

For tracking we usually would set some camera with sysid/compid in control which then does the tracking but I can see how that doesn't answer pointing at a sysid and pointing at home.

Should we extend MAV_CMD_DO_GIMBAL_MANAGER_PITCHYAW for these use cases or should we add one more message, e.g. MAV_CMD_DO_GIMBAL_MANAGER_TRACK, and then an enum listing the various tracking options.

And that would be different from MAV_CMD_CAMERA_TRACK_POINT which is more for a camera.

[olliw42]

@julianoes

this isn't meant sarcastic at all, but you may understand that I kind of feel confirmed in what I am saying since ca 2 years and not getting through with it

• maybe you want to finally revise your position concerning RC targeting/control, not only for gimbal device but also gimbal manager
• maybe you want to think through what I call "super modes" and what is essentially mount mode, only put onto a consistent and extensible concept

while your suggestions may somehow may allow to achieve the things, it's obviously of the kind of from-behind-through-the-chest type of workaround. As I always do, I wonder, why not putting it onto solid feet

sorry for being a pain

So to signal that RC is in control, I use the sysid_primary_control and compid_primary_control of the autopilot (for PX4 that's by default 1, 1) which means that itself is in control, and itself has an RC receiver. I understand this is a bit of a kludge...

and there might be other reasons it may be set to itself in control, like missions, .... how then to discriminate what the intention is

For tracking we usually would set some camera with sysid/compid in control which then does the tracking but I can see how that doesn't answer pointing at a sysid and pointing at home.

but most cameras out there can't be set with sysid/compid in control and then do the tracking, so not even this would be an answer in many cases

Should we extend MAV_CMD_DO_GIMBAL_MANAGER_PITCHYAW for these use cases or should we add one more message, e.g. MAV_CMD_DO_GIMBAL_MANAGER_TRACK, and then an enum listing the various tracking options.

And that would be different from MAV_CMD_CAMERA_TRACK_POINT which is more for a camera.

so you end up with a message/command for every case even though all it needs is a mount_mode (aka super mode) value

hmhmhm

[rmackay9]

@julianoes @olliw42,

OK great thanks. I'll go and think about it a bit more but I think we need to retain the concept of gimbal mode rather than trying to squish it into the existing messages or flags. I think we will probably need a set-mode command and also reporting back somehow from the gimbal manager on what the gimbal's mode is.

[olliw42]

I think we need to retain the concept of gimbal mode rather than trying to squish it into the existing messages or flags. I think we will probably need a set-mode command and also reporting back somehow from the gimbal manager on what the gimbal's mode is.

that's my opinion too, except that I made "gimbal mode" a bit more general, calling it "super mode" (which is admitedly a silly name, pl feel free to invent a better one).

I wonder if you might find it usefull to read the part on "super modes" in here http://www.olliw.eu/2020/mavlink-gimbal-protocol-v2/. Sadly, there is no direct link to the relevant location, so please scroll down to the paragraph which starts with "Remains to clarify what the super modes are." (seraching in the web page for super modes should lead you quickly)

I wonder if this isn't exactly what you actually want.

[julianoes]

Alright, I'm open for suggestions :smile:.

[KylePreuss]

Without the concept of these modes how does a GCS know if the gimbal is supposed to be looking at a lat/lon (i.e. MAV_MOUNT_MODE_GPS_POINT) vs being controlled by angles or slews? It sounds like it could be accomplished thru additional GIMBAL_MANAGER_FLAGs or a mode field. Either way it may be useful.

Along those lines I'm curious how the GCS might request / see the last commanded values for those modes e.g. get the lat/lon from the last MAV_CMD_DO_SET_ROI_LOCATION but that is another issue.

@olliw42 FYI your link on the gimbal protocol has gone down