mavlink / MAVSDK

API and library for MAVLink compatible systems written in C++17
https://mavsdk.mavlink.io
BSD 3-Clause "New" or "Revised" License
615 stars 503 forks source link

MAVSDK timeout tuning #1441

Closed JonasVautherin closed 2 years ago

JonasVautherin commented 3 years ago

It seems to me that the timeout used in MAVSDK is a bit low for bad links (I'm thinking about a slow connection to a cloud simulator), resulting in a lot of timeouts (which makes it difficult to e.g. upload a reasonably-sized mission).

@julianoes: is there a global timeout that can be tuned, and what would it imply if we increased it? Like do we want to have an option to set the timeout (or choose the link speed), or would it be fine to just increase the timeout for everybody?

From my tests uploading a big mission to a cloud simulator, each waypoint took an average 0.4s to upload... I have heard of issues when uploading from a device emulator (iOS/Android), so I wonder if that maybe went above 0.5s, triggering the timeouts?

julianoes commented 3 years ago

I added this a while ago, give it a try: https://github.com/mavlink/MAVSDK/blob/f7e96f3579c4d199e8a28197568409cfbd65bfb4/src/core/mavsdk.h#L271-L279

reedev commented 3 years ago

Hi,

First post here so I hope this is the right location...

I would like to get this working too and I am willing to put effort in this to achieve this. I am currently trying to upload a mission via a serial connection (holybro telemetry radio v3) to a pixhawk 4 and keep getting the timeout. I am using mavsdk v0.40.0

Can you give my some guidance?

Thanks, Raymond

JonasVautherin commented 3 years ago

@reedev doesn't the set_timeout_s function above do what you want? Haven't used it yet, but I think it's exactly what I want

reedev commented 3 years ago

no, I put the line mavsdk.set_timeout_s(5.0); right after the allocation of mavsdk and within the 5 seconds I get the message 'Mission upload failed (Timeout), exiting'

julianoes commented 3 years ago

@reedev this would mean something with the other side or the communication goes wrong. Please create a new issue and describe how to reproduce the problem.

reedev commented 3 years ago

Sure, that's why I opened the other issue, so that I can chip in myself and find details. I will get back on this.

reedev commented 3 years ago

That's interesting: I went back in time and I could upload the mission via the fly_mission.cpp from v0.22.0 Although with some timeouts 28 mission items were uploaded. Double checked with QGC and I see the mission near the Milchbuck-Tunnel

I will proceed... (when time allows)

reedev commented 3 years ago

(intermediate update) this is rather interesting: v0.24.0 works as expected while v0.25.0 does not anymore. The interface has changed between these versions too so it seems a rather large change. Still going on...

julianoes commented 3 years ago

We have an integration test for lossy mission transfers: https://github.com/mavlink/MAVSDK/blob/main/src/integration_tests/mission_transfer_lossy.cpp

So best would be if you could change that so you can reproduce your issue.

julianoes commented 3 years ago

From v0.24.0 to v0.25.0 I completely rewrote the mission upload, so it's possible I got something wrong.

JonasVautherin commented 3 years ago

So best would be if you could change that so you can reproduce your issue.

I second that! It would be very interesting if you had an integration test that fails for you, so we could try to reproduce on our side!

reedev commented 3 years ago

Sure! I will do my best and need to dive into this first to get familiar with the integration tests.

julianoes commented 3 years ago

This should be enough:

cmake --build build/default -j8 && build/default/src/integration_tests/integration_tests_runner --gtest_filter="SitlTest.MissionTransferLossy"
ankitatcodingnebula commented 3 years ago

@julianoes I use MAVSDK-Java in my project and I had the timeout issues with upload and download over high latency radio link. I compiled a new version of MAVSDK by setting the default timeout as below in mavsdk.h

static constexpr double DEFAULT_TIMEOUT_S = 5;

I can confirm that the upload mission timeout is fixed but this doesn't work for download mission api. Is there a different timeout setting for download mission?

Also just an FYI, I use blockingGet() api in java sdk to download and upload missions. Not sure if this is relevant here.

Please suggest.

beenjamminnallday commented 3 years ago

I am using the recently released MAVSDK Python 0.19.0, and similar to what is described above I increased the set_mavlink_timeout(timeout_s) function to 5 and 10 secs, It doesn't work with more than 1 waypoint. Uploading a mission via a serial connection (holybro telemetry radio v3) to a pixhawk 4 unfortunately results in a timeout.

julianoes commented 3 years ago

5 or 10 seconds seems way too high, it likely won't work because PX4 on the other side still assumes a timeout much lower, around 0.5 seconds. What is the ping/latency between MAVSDK and PX4 for your setup?

BeenJamminAllDay commented 3 years ago

I might be wrong but I assumed the timeout function told PX4 what timeout to expect? From your answer, I gather I would have better results if put the timeout at 1 or 2 secs?

I haven’t checked the latency between MAVSDK and PX4 but since I’m using radios to communicate, it shouldn’t be too high.

reedev commented 3 years ago

Quick update from my side. I noticed the recent timeout related changes and tried out fly_mission and the result is still the same, could not upload a mission. I will check out the integration test asap...

reedev commented 3 years ago

v0.42.0 I tested with a Herelink / Pixhawk4 / PX4 setup with a different result: the autopilot is not found. I also tested with sitl with Gazebo and that's working as expected.

JonasVautherin commented 3 years ago

I might be wrong but I assumed the timeout function told PX4 what timeout to expect?

No, it's really on the MAVSDK side: it tells MAVSDK which timeout to use. You are right that it would make sense to synchronize that value with PX4, but MAVLink does not provide that, and I don't know what it would imply on the PX4 side (and how one would handle the case where PX4 does not support it).

JonasVautherin commented 2 years ago

Let's close that, it's been inactive for a long time. Feel free to reopen if necessary, or create a new issue if the timeout has a bug.