How do I get an image from the MavSDK?

[takata1021]

Please tell me how I want to fly gazebo's camera drone (iris_depth_camera) and operate the camera from the MavSDK. Even if drone.camera.take_photo () is executed, it will be an error.

[takata1021]

QGroundControl uses the Camera plugin (libgazebo_camera_manager_plugin.so) to access the Gazebo camera. According to the guide this supports the Mavlink command. In other words, the MavSDK also has access to the Gazebo camera via Mavlink. Is this a bug?

[Jaeyoung-Lim]

@takata1021 This is not a bug

The depth camera is simulating a rgb-d camera, which is typically used for obstacle avoidance. Thereore it does not use the camera manager plugin.

Yoi can try with typhoon_h480 if you want to simulate a camera manager over mavlink

[takata1021]

thank you. I used typhoon_h480 but couldn't access the Gazebo camera (libgazebo_camera_manager_plugin.so) from the Mav SDK. In the user guide, the UDP port for offboard apps is 14540. So the Mav SDK app uses the UDP port: 14540 to connect to the drone, but this doesn't allow access to the Gazebo camera. You can access the Gazebo camera by setting the UDP port of the Mav SDK app to 14550. 14550 is a port for GCS. Is this movement correct?

[Jaeyoung-Lim]

@takata1021 This is where the mavlink port for the camera manager is available : https://github.com/PX4/PX4-SITL_gazebo/blob/25138e803ee8525ee5fe4e6d511506e88e3f819c/models/typhoon_h480/typhoon_h480.sdf.jinja#L418

You can check the generated sdf file to see which port it is using

[takata1021]

I couldn't understand. I want you to tell me.

The camera plugin for typhoon_h480.sdf is UDP port 14530.

<plugin name = "CameraManagerPlugin" filename = "libgazebo_camera_manager_plugin.so">
             <robotNamespace> typhoon_h480 </ robotNamespace>
             <interval> 1 </ interval>
             <width> 3840 </ width>
             <height> 2160 </ height>
             <maximum_zoom> 8.0 </ maximum_zoom>
             <video_uri> udp: //127.0.0.1:5600 </ video_uri>
             <system_id> 1 </ system_id>
             <cam_component_id> 100 </ cam_component_id>
             <mavlink_cam_udp_port> 14530 </ mavlink_cam_udp_port>
</ plugin>

I can't access this camera plugin from the Mav SDK. I'm using camera.py at https://github.com/mavlink/MAVSDK-Python/tree/main/examples.

How do I match the UDP camera port (14530)? Please tell me how to move it.

[Jaeyoung-Lim]

@takata1021 Why do you need to move it? The camera port is at 14530

[takata1021]

Why do you need to move it?

This is necessary because we are developing an app that shoots with the camera from an offboard app via the Mav SDK. I tried it on Gazebo simulator using camera.py and typhoon_h480 at https://github.com/mavlink/MAVSDK-Python/tree/main/examples. But that doesn't work. That's why I'm asking how to do it.

Does the Gazebo simulator support launching typhoon_h480 and shooting from MAVSDK using the camera class? If you support it, please tell me how to do it.

The camera port is at 14530

How do I get from the Mav SDK to the square port 14530?

[takata1021]

I also tried this but it doesn't work.

[Jaeyoung-Lim]

@takata1021 The python code is using udp port 14540: https://github.com/mavlink/MAVSDK-Python/blob/edd21f4d95a5a37ca1a24fd7ffb97da5bfb9d86d/examples/camera.py#L11

You need to use udp port 14530

[takata1021]

I was able to access udp port 14530, but the camera class fails. Please tell me the mistake. I am using the following code.

#!/usr/bin/env python3

import asyncio

from mavsdk.camera import (CameraError, Mode)
from mavsdk import System

async def run():
    drone = System()
    await drone.connect(system_address="udp://:14540")

    print("Waiting for drone to connect...")
    async for state in drone.core.connection_state():
        if state.is_connected:
            print(f"Drone discovered!")
            break

    print("Waiting for camera to connect...")
    cameratest = System()
    await cameratest.connect(system_address="udp://:14530")

    async for state in cameratest.core.connection_state():
        if state.is_connected:
            print(f"camera discovered!")
            break

    print_mode_task = asyncio.ensure_future(print_mode(drone))
    print_status_task = asyncio.ensure_future(print_status(drone))
    running_tasks = [print_mode_task, print_status_task]

    await asyncio.sleep(10)
    print("Setting mode to 'PHOTO'")
    try:
        await cameratest.camera.set_mode(Mode.PHOTO)
    except CameraError as error:
        print(f"Setting mode failed with error code: {error._result.result}")

    await asyncio.sleep(10)

    print("Taking a photo")
    try:
        await cameratest.camera.take_photo()
    except CameraError as error:
        print(f"Couldn't take photo: {error._result.result}")

    await asyncio.sleep(10)

    # Shut down the running coroutines (here 'print_mode()' and
    # 'print_status()')
    for task in running_tasks:
        task.cancel()
        try:
            await task
        except asyncio.CancelledError:
            pass
    await asyncio.get_event_loop().shutdown_asyncgens()

async def print_mode(drone):
    async for mode in drone.camera.mode():
        print(f"Camera mode: {mode}")

async def print_status(drone):
    async for status in drone.camera.status():
        print(status)

if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(run())

[Jaeyoung-Lim]

@takata1021 I think your question was answered in https://github.com/mavlink/MAVSDK/issues/1692

Also, the code you showed is still using udp port 14540

[takata1021]

I think your question was answered in https://github.com/mavlink/MAVSDK/issues/1692

Is it okay that the Gazebo camera cannot be accessed from the Mav SDK? Please note in your guide that the Mav SDK does not support gazebo cameras.

Also, the code you showed is still using udp port 14540

just await cameratest.connect (system_address = "udp: //: 14530") didn't work properly (connection doesn't complete). Do await cameratest.connect (system_address = "udp: //: 14530") after await cameratest.connect (system_address = "udp: //: 14540") to complete the connection. But the camera doesn't work properly.

If the Mav SDK doesn't support Gazebo cameras, won't it work when connected to UDP14530?

What are your instructions trying to do?

[Jaeyoung-Lim]

I think your question was answered in https://github.com/mavlink/MAVSDK/issues/1692 Is it okay that the Gazebo camera cannot be accessed from the Mav SDK? Please note in your guide that the Mav SDK does not support gazebo cameras.

The guide mentions support with mavlink and not mavsdk.

If the Mav SDK doesn't support Gazebo cameras, won't it work when connected to UDP14530? What are your instructions trying to do?

Not sure what you mean, you can speak to the gazebo camera manager through 14530. If you want to check the code, you can look at the behavior yourself: https://github.com/PX4/PX4-SITL_gazebo/blob/master/src/gazebo_camera_manager_plugin.cpp

On why mavsdk does not connect to the camera manager, I am not sure. One thing you can try is to switch the camera port https://github.com/PX4/PX4-SITL_gazebo/blob/25138e803ee8525ee5fe4e6d511506e88e3f819c/models/typhoon_h480/typhoon_h480.sdf.jinja#L418

to 14540 and see if it works. I could use it this way for my application, but not using mavsdk

[takata1021]

thank you. I understand what you are saying. You told us that you can connect to UDP 14530 via Mavlink and operate with MavLink messages.

You don't use the Mav SDK, do you. I want to control the camera from the Mav SDK.

o 14540 and see if it works. I could use it this way for my application, but not using mavsdk

This didn't work either. Is it possible to get a camera image from the MavSDK using the Gazebo simulator? PX4 / PX4-Autopilot # 19106.

[Jaeyoung-Lim]

You don't use the Mav SDK, do you.

@takata1021 I do, but I don't think the camera example you are trying to use was written for gazebo SITL.

Lets continue the discussion in https://github.com/mavlink/MAVSDK/issues/1692 This is issue related to mavsdk and not PX4

[takata1021]

Thank you for everything. That helps a lot.