New Gimbal Interface

[robertlong13]

Still a bit to do, but I have some stuff working. It's going to be a massive PR, so I want to get some of this out there to start review early.

• I have a test setup using RealFlight that I will need to document so anyone can play around with this.

• I have a lua script I need to publish that sends VIDEO_STREAM_INFORMATION messages so Planner can detect the URIs for the video.

• ArduPilot's CAMERA_FOV_STATUS message has a bug that causes the wrong units to be used for alt. Needs fixing.

• Most of the buttons and right-click options do not work yet

• Need a settings page so people can adjust the stuff like keybindings

• Things you can do

  • WASD to slew. Hold shift to go faster, ctrl to go slower.
  • Hold E to zoom in, hold Q to zoom out.
  • Clicking sets the POI to the lat/lon of the point you clicked, though more options are intended. (THIS ONLY WORKS IF YOU HAVE PATCHED CAMERA_FOV_STATUS to send the correct units)
  • Alt-F to take a picture
  • Alt-R to toggle recording
  • L to toggle between lock and follow
  • N to go neutral
  • H to point at home

(Terrible gif quality, had to really squeeze it down to fit in a github post, so there are a ton of artifacts; these obviously aren't in the real thing)

[rmackay9]

This is really great to see! Thanks for also finding the issue with FOV_STATUS. We added support ages ago but no GCS has used it yet so I guess we missed the bug.

[Davidsastresas]

Very neat! :)

[IuliuNovac]

How did you manage to get the simulation with a video feed up and running, is that gazebo?

[robertlong13]

How did you manage to get the simulation with a video feed up and running, is that gazebo?

I use RealFlight. I'm sure the same thing could be done in Gazebo, but I don't know my way around it nearly as well. You need this model and this lua script. You also need to be running on AP master (until the FOV message bug fix gets backported to 4.5)

I use a gstreamer pipeline to grab a cropped area of a monitor. I toyed with using OBS to stream as RTSP, but that introduced more than a second of latency.

dx9screencapsrc x=10 y=60 width=640 height=360 monitor=1 ! videoconvert ! video/x-raw,format=BGRA ! appsink name=outsink sync=false

Adjust the monitor, x, y, width, and height for your setup.

If you configure your autopilot to provide VIDEO_STREAM_INFORMATION messages, with gst://[pipeline] as the URI, this new UI can grab the pipeline from there automatically (it doesn't auto connect to it yet though; that's a planned feature).

The airport in this video was Pendleton Airport (KPDT), but I don't know if I can share my RealFlight model of that airport publicly. Instead, I'll share Base INFOCA, which has the side benefit of being within the real terrain model RealFlight pulled from Spain, so all the terrain actually lines up: Base INFOCA_AP.zip. The downside is that I went WAY overboard manually adding one metric crapload of trees to it. Disable trees with View > Scenery > Trees to disable them if your computer is choking on them. Launch SITL with -O 37.412620,-3.075734,1052.0,0

[robertlong13]

This PR is getting close to ready. All buttons and (almost) all keybindings are implemented, and I have a nice settings page where you can customize every keybinding.

Still needed:

• Click to track (need a testing setup for this one)
• Auto connect to first video stream if found
• Consider making a way to embed this on the flightdata page with minimap, and have the ability to swap between map with mini video and video with mini map.

[meee1]

need to tweak for android/apple - currently failing. system.drawing cant be used easerly....

[robertlong13]

Tracking added

[robertlong13]

Added new ways to display the new control

Full Sized. In this mode, there is a mini map by default, but it can be hidden with a button in the right-click menu.

Mini

Mini and full-sized can be swapped at any point with the "swap with map" when right clicking on the video.

And pop-out works exactly as before (except now it remains always on top, which I think is always better, you can always intentionally minimize it if it's in the way).