Emulate Camera in MavSdk

[tank104]

Hi, I am trying to figure the approach to emulating a camera using the MAVSDK.

Ultimately what I am trying to achieve is when missions run, and there is a camera action, I want to take a picture in my Sim based off this action (I am using Airsim).

I see with MavLink I effectively need to send a heartbeat for the camera, and then the camera information - https://mavlink.io/en/services/camera.html. But I can't figure out how I do that with MavSdk.

Is MavSdk designed to be used as a Camera controller for a new camera (i.e. not just control an exisiting MavLink camera)?

To perhaps make things more complex I am using MavSdk server (i.e. via Grpc).

[JonasVautherin]

I think you want to check the camera_server module.

• camera is for controlling the camera (e.g. from the ground)
• camera_server is to implement a camera that will be controlled by camera

Does that make sense?

[tank104]

That does make sense - I couldn't find it in the documentation - https://mavsdk.mavlink.io/main/en/cpp/api_reference/

I also didn't have it in my proto branch (which I thought was based on the latest MavSdk release).

So its all supported by the MavSdk Server, just undocumented?

[JonasVautherin]

I'm confused. You mean this one? Seems like it is in mavsdk_server.

[tank104]

Ahh thanks! I missed it because it’s not in the menu unlike other servers. Should have looked harder but I just assumed it didn’t exist.

Will look tomorrow as to why it wasn’t built from the proto

[tank104]

Yeah looks like it’s in submenu under Camera, and not in top level like other servers. Not sure if it’s possible for me to fix that?

[tank104]

@JonasVautherin back at computer now - I still think there is an issue. That latest MavSdk release is v1.4.9 The proto folder on this links to proto @ 743aa1d

Within that commit there is no camera_server https://github.com/mavlink/MAVSDK-Proto/tree/743aa1d24094db30ac3ed503d9552fb9c49ef4c4/protos

[JonasVautherin]

Oh, maybe that will come with MAVSDK v2. So you may have to build MAVSDK from source in the meantime :thinking:

[tank104]

Ahhh ok that is making more sense now - and probably why it wasn't in the main documentation either then.

I will build from sources and see how I go.

Any ETA on V2 - had a quick look and couldn't see anything.

[tank104]

@JonasVautherin - sorry to bother you again - just wondered if there were any examples on using camera_server? I have built the latest MAVSDK so have the latest server that should support it, but its just not working.

Normally examples help in this case, but have googled and cannot find anything.

[tank104]

Found this camera_take_photo.cpp, but ideally want an example that would work using the Masvsdk_server

[JonasVautherin]

Not that I know of, unfortunately. When you get it working, it would be really cool if you could contribute an example though :blush:

[tank104]

Definietly!

[tank104]

I have managed to get it working for Photo Intervals on mission items (see below rudimentary c# example). Not working for individual TakePhoto mission items though, so working through that.

mavSdkSystem.CameraServer.TakePhoto().Do(
  takePhotoInt =>
  {
    _logger.LogInformation("CameraServer.TakePhoto - {takePhotoInt}", takePhotoInt);
  },
  exception =>
  {
    _logger.LogError("CameraServer.TakePhoto error - {exception}", exception);
  }).Retry().Subscribe();

[tank104]

@JonasVautherin do you have any idea why for Mission Items CameraAction "StartPhotoInterval" triggers TakePhoto() based on interval, but CameraAction "TakePhoto" doesn't trigger anything. Is there logic for that that I cannot find online?

[tank104]

I can kind of get around it by adding a mission item where I do StartPhotoInterval then StopPhotoInterval 1s later (and have 1s interval), although it doesn't seem entirely accurate - and can sometimes send more than one TakePhoto request.

I am now wondering after spending a lot of time on this if I am doing it wrong - and should just handle the camera separate from MavLink itself, and just control it seperately.

[JonasVautherin]

do you have any idea why for Mission Items CameraAction "StartPhotoInterval" triggers TakePhoto() based on interval, but CameraAction "TakePhoto" doesn't trigger anything. Is there logic for that that I cannot find online?

I don't :thinking:. Would be nice to debug it. My understanding is that both StartPhotoInterval and TakePhoto use the same MAVLink message: MAV_CMD_IMAGE_START_CAPTURE.

The processing is happening here. More specifically, the single capture is here. Maybe there is a bug there?

[julianoes]

There are two APIs. The trigger API which is usually used with the autopilot which then triggers an image. And the camera capture API that MAVSDK uses which is sent directly to a camera that implements the triggering.

[JonasVautherin]

Oh, so the autopilot does not use MAV_CMD_IMAGE_START_CAPTURE to trigger the camera?

[julianoes]

I doesn't know what to do with that, pretty sure, except forwarding it to "a camera".

[julianoes]

I think we mostly need to finish the work for a camera server example.