search

svpcom / wfb-ng

WFB-NG - the next generation of long-range packet radio link based on raw WiFi radio
https://docs.px4.io/main/en/tutorials/video_streaming_wifi_broadcast.html
GNU General Public License v3.0
1.03k stars 242 forks source link

[DESIGN] how to bypass up&down camera control mavlink messages #279

Open maxlapshin opened 1 year ago

maxlapshin commented 1 year ago

Feature proposal

Mavlink has some builtin messages for controlling camera and reporting its status.

If proper messages will be passed through wfb, then we will be able to get following features:

  1. controlling camera in QGC and with remote control just like on "casual drone"
  2. standardized controlling of camera on the drone in the same standard manner that mavlink offers

So there are two separate tasks:

  1. pass status messages from the drone to GS where QGC will be able to handle them
  2. pass control messages from GS to some software on the drone

Downside camera status messages

WFB has camera functionality out of the box, so it may look rather convenient if it will report camera existence out of the box and pass rtp udp url to GS. However, wfb is not a complete turn-key solution, it is a technology, so I suppose that this functionality should be designed as a preconfigured opt-in feature, rather than opt-out.

So as for me, sending CAMERA_INFORMATION by default is a very good idea, but it should be opt-in with some default configuration, because user can decide to create his own information message.

Upside camera control messages

QGC can send commands from a joystick (or a headless multi-drone management system) to the drone. Question is: who is supposed to handle them?

When fpv-camera is launched, it may be a good idea to bypass messages like MAV_CMD_DO_DIGICAM_CONFIGURE to the gstreamer, capturing video.

When wfb is launched in an ethernet-proxy mode (when IP camera on the drone is sending video packets to the wfb), it may be better to launch some proxy that will translate mavlink commands to onvif commands.

Here I see a question: what is the proper way to distribute mavlink messages from the ground? Some of them should be targeted to a flight controller, some of them must be handled by some user provided software. Maybe the best option is a full mirror? FC knows how to fly and my own software knows how to control gimbal, attached to a raspberry pi?

svpcom commented 1 year ago

wfg-ng now have full-mirror proxy (localhost:5760) and you can write standalone program which will subscribes to mavlink messages and do some actions. Also there are call_on_arm and call_on_disarm hooks. For example I use it for starting/stopping video recording when drone arms/disarms.

There are several joystick commands which QGC can issue: drone control and gimbal control. All of them are handled by autopilot, but PX4 has virtual gimbal subsystem that can pass this messages to physical gimbals (for example storm32) which is connected to autopilot UART. But number of this commands are limited and hardcoded into QGC. For example there is no "zoom in/zoom out" commands. So for these tasks I use standalone python program which poll /dev/input/js0 and call camera url directly via ip-tunnel.