search

mavlink / mavros

MAVLink to ROS gateway with proxy for Ground Control Station
Other
879 stars 989 forks source link

Implementation of image streaming via MAVLink #30

Closed mhkabir closed 10 years ago

mhkabir commented 10 years ago

http://qgroundcontrol.org/mavlink/image_transmission_protocol

Would be great if the above could be implemented such that ROS Images could be send down via Mavlink to GCS. Mavros could act as a ROS<>Mavlink video bridge.

vooon commented 10 years ago

Transfer video from MAVLink in ROS is not a bad feature, and it is easy to implement. But I did not want to add anything to udp bridge, and there is no suitable api for plugins.

What GCS support video display? Possibly rqt better choice.

mhkabir commented 10 years ago

QGroundControl supports video streaming from Mavlink. It would be very useful if we could get image from ROS into the GCS for some custom computer vision applications, etc.

vooon commented 10 years ago

Now I do not want map mavros to mavlink udp, not all GCS are able to work with systems (same sysid, several compid).

But I think you can send video if you do not use the built-in bridge. Instead, use ros_udp and publish video messages via /mavlink/from.

mhkabir commented 10 years ago

Yeah, using ros_udp would be good as we could simply start another UDP link to the GCS.

It would be like this : USB camera > ROS/Image processing/ etc.> ros_udp > GCS

How would you implement this? Would it be easy?

vooon commented 10 years ago

Common type for image streaming is sensor_msgs/Image, so camera node will send images. Processing node also send images in ROS format and new extra node translates any sensor_msgs/Image to Mavlink messages.

I don't know how easy it is, but look image_transport wiki page.

mhkabir commented 10 years ago

Check out this PR for mavconn, where image and visited stream is implemented. It should give you a good idea of what we can do : https://github.com/pixhawk/mavconn/pull/4/files

mhkabir commented 10 years ago

@vooon I think we can get this working soon enough. I've got a basic system structure ready :)

mhkabir commented 10 years ago

@vooon Can you give me example of how to send a mavlink message to GCS in a plugin. A code snippet will do.

vooon commented 10 years ago

You can't send to GCS from plugin. But you can use external gcs bridge (ros_udp) and send GCS stream in /mavlink/from topic.

mhkabir commented 10 years ago

Can you add API for sending to GCS using plugin? PLEAAASSEE!! :D

On Mon, Jul 28, 2014 at 10:49 PM, Vladimir Ermakov <notifications@github.com

wrote:

You can't send to GCS from plugin. But you can use external gcs bridge ( ros_udp) and send GCS stream in /mavlink/from topic.

— Reply to this email directly or view it on GitHub https://github.com/vooon/mavros/issues/30#issuecomment-50368695.

vooon commented 10 years ago

Not now. I think mavros should be transparent(no their streams) for MissionPlanner.

mhkabir commented 10 years ago

Okay....

mhkabir commented 10 years ago

@vooon Can you give me a simple example on how to write to /mavlink/from ?

vooon commented 10 years ago

Look there: https://github.com/vooon/mavros/blob/master/src/ros_udp.cpp#L39

vooon commented 10 years ago

I think you can write specialized version on ros_udp instead of sending images to ros topic. And i think it is faster.

It must use image_transport and then convert image to mavlink messages and mix it with data from regular topics.

vooon commented 10 years ago

As soon as i understand image transmission i create special gcs_bridge, but place for that must be in that package: https://github.com/vooon/mavros_extras

mhkabir commented 10 years ago

I can help you with that, let me know what you need help in. Basically, the main thing is, take image from topic, store in CV::Mat, convert to JPG , split into smalled chunks and send via ENCAPSULATED_DATA messages.

On Wed, Jul 30, 2014 at 9:08 PM, Vladimir Ermakov notifications@github.com wrote:

As soon as it understand image transmission i create special gcs_bridge, but place for that must be in that package: https://github.com/vooon/mavros_extras

— Reply to this email directly or view it on GitHub https://github.com/vooon/mavros/issues/30#issuecomment-50632684.

vooon commented 10 years ago

Seems that documentation wiki is outdated, i look into px4flow firmware, and in QGC sources. It just send handshake and then bunch of data messages.

vooon commented 10 years ago

Just finished gcs_image_bridge. It works, but QGC drops some frames:

src/uas/UAS.cc 2093 Loading data from image buffer failed!
mhkabir commented 10 years ago

Hmm. Okay I will look into it and get back if I can patch a fix.

This might be a QGC problem too... Need to test. On Jul 31, 2014 1:48 AM, "Vladimir Ermakov" notifications@github.com wrote:

Just finished gcs_image_bridge. It works, but QGC drops some frames:

src/uas/UAS.cc 2093 Loading data from image buffer failed!

— Reply to this email directly or view it on GitHub https://github.com/vooon/mavros/issues/30#issuecomment-50673598.

vooon commented 10 years ago

I still get this QGC error, but it works (i can see video from laptop camera). Also image_pub shows image stream without problems (i build chain uvc_camera → gcs_image_bridge → mavros → rqt_image_view).

Implemented in https://github.com/vooon/mavros_extras I close this issue.