Simultaneous RC and Virtual Joystick Control

[MattAuto28]

With our GCS, we operate via virtual joystick. At times, the data link can be lost. This means it should fall to the data loss failsafe. However, as autonomous failsafe methods can themselves be flawed when there are very specific flight routes (flying through buildings, under tree cover, etc), we prefer to take over with RC.

To avoid relying on the data loss failsafe, when the data link is lost, we immediately shut down the GCS, and power on an RC transmitter to hopefully be able to take manual control of the drone before it smashes itself to pieces. This is an imperfect solution.

The preferable way for PX4 to operate would be able to allow simultaneous RC and Virtual Joystick connection, with a tunable parameter determining who has priority over the core flight characteristics (pitch/yaw/roll/throttle). In this way, all additional RC channels would still be able to work at all times (for actions like killswitch, parameter tuning, arm/disarm, flight mode), while you're using virtual joystick. If something were to go wrong you flip an RC switch to give priority to your RC controller and can rescue the drone.

This small change would save many thousands of dollars worth of drones!

[dagar]

This comes up periodically, and basically what's missing is someone willing to work through the details of a selection mechanism that's no worse than the current setup (only one manual input selected at boot).

Previous suggestions would have resulted in things like control jumping from Joystick to RC or vice versa. Whatever the solution, it needs to be safe, well understood, and involve operator intent for retaking control.

[MattAuto28]

Previous suggestions would have resulted in things like control jumping from Joystick to RC or vice versa. Whatever the solution, it needs to be safe, well understood, and involve operator intent for retaking control.

Do you have any thoughts on the mechanism proposal that I laid out?

[dagar]

Do you have any thoughts on the mechanism proposal that I laid out?

I don't see how a simple priority is safe in most situations. Imagine flying with RC in a manual mode, losing it briefly, and control immediately jumping to the Joystick (probably centered). How will the operator know what's going on?

I think it needs some notion of primary manual input combined with a mechanism to explicit switch or retake control. If both RC and Joystick are connected then only one of them can be considered the active primary. If you lose your primary it should be handled like a normal loss of RC failsafe. To switch into manual control on the other link you should have to do something to request and retake control.

Does that make sense?

[MattAuto28]

Does that make sense?

I understand your thoughts on that, and why it may not be an ideal case for everyone - but if it's based off parameters then only the people who understand it and are prepared for it should be using it. As a midway point, even just having the priority being a tunable parameter and always allowing the auxiliary RC channels would be enough for a safer flight. I cannot see a reason why ALL of the RC channels are disabled if only four of them are being replaced by virtual joystick.

Right now, lacking an RC failsafe makes flying with virtual joysticks illegal at many testing sites, and makes it particularly difficult to certify for commercial operations.

[dagar]

... and always allowing the auxiliary RC channels would be enough for a safer flight.

Now that I think of it, this unrelated PR should make the RC switches still function independently of the primary manual control. https://github.com/PX4/Firmware/pull/9712

After v1.9.0 goes out I'll rebase https://github.com/PX4/Firmware/pull/9712 and get it merged.

Right now, lacking an RC failsafe makes flying with virtual joysticks illegal at many testing sites, and makes it particularly difficult to certify for commercial operations.

Then let's keep the conversation going and see if we can converge on a solution that both enables the desired use case, but doesn't create more safety holes in the process. All we need is to think of a way for a user to retake control both from the RC side and then from the Joystick side. RC side could probably be a switch. Joystick side perhaps a mavlink message. https://mavlink.io/en/messages/common.html

[zozo2020]

The best way, as I see it, is to add a new flight mode, "virtual joysticks", that control only pitch+yaw+roll (x,y location without z), and keep the rc logics as is

[MattAuto28]

The best way, as I see it, is to add a new flight mode, "virtual joysticks", that control only pitch+yaw+roll (x,y location without z), and keep the rc logics as is

Having it as a flight mode is not a great solution, as you would have to create Virtual Joystick variants of every manual/assisted flight mode. Just having a single parameter dictating who has control would be easy to toggle from another RC channel.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[TSC21]

Still relevant

[bozkurthan]

Hello, I'm interested in multiple source of manual_control_setpoint.

Actually, I implemented some code to QGC. The mechanism works between virtual thumb joystick and normal joystick that is connected from USB. I added button that allows virtual joystick control to QGC. But there is a trick for this; I assumed Joystick a real RC. It's be like Joystick works as a safety choice. I mean I control PX4 from virtual joystick. But if something goes wrong, then Joystick takes control of drone. For this, if I change location of Joystick's sticks, then virtual joystick stops sending manual_control_input, then normal joystick handles control. But I didn't change COM_RC_IN_MODE. I just play with joystick_enabled.
The whole steps tested in simulation. So, I wanted to test in real flight. Before this, I wasn't know the PX4 side accepts only one manual_control_setpoint source.

I know that the mechanism choices only first source.

By @dagar

This comes up periodically, and basically what's missing is someone willing to work through the details of a selection mechanism that's no worse than the current setup (only one manual input selected at boot).

Is there any way to accept multiple manual control setpoint or choice different instance of source.

For example there is 2 source of manual_control_setpoint. First one is a RC, second one is the QGC.

And there is only 1 source of input_rc.

Last this is conf of COM_RC_IN_MODE.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. Thank you for your contributions.

[junwoo091400]

Is this not solved by https://github.com/PX4/PX4-Autopilot/pull/17404?