Mavproxy starts up with only garbage packets - restarts fine

[arsenixprime]

I have a number of vehicles I have built which run Raspberry Pi's hooked up via the serial port to a Pixhawk. Mavproxy is configured to start at boot. Some non-zero percentage of time mavproxy starts up in what I call "garbage mode" and just pumps out noise packets - never connects to autopilot. Restarting mavproxy it comes up properly the second time every time. It seems almost like the baud rate isn't configured properly sometimes. I have dug and dug to see if any other process is opening the serial port and mucking with things, but I haven't found anything. Likewise I have played with the startup timing extensively and it makes no difference.

Has anyone experienced this? Any tips? Considering patching mavproxy to have a "restart on excessive noise" function, but that seems like a bandaid.

[stephendade]

I've not heard of this happening before - and many people run MAVProxy on their Pi's

Is there a way you can reliably replicate the issue? -Have you removed modemmanager on the Pi's? -Does the issue occur if the Pixhawks are connected directly to a PC? -Does this issue only occur with particular Pixhawks, or all of them? -What APM version and Pixhawk type are you using?

[rozzin]

I can replicate this, but not reliably.

By the characteristics of the garbage, it looks like the baud rate is somehow actually set to 9600 despite mavproxy trying to set it to 57600--and despite the fact that stty reports the port's configured baud rate as 57600. If I `change' it to 9600 with stty, the noise mavproxy reads from the remains characteristically the same; if I then use stty to set it back to 57600, then mavproxy stops seeing noise and starts seeing the expected valid MAVLink stream.

The autopilot is hooked up to ttyAMA0 on the raspi, which I believe is the `good' UART on the raspi3 (not the mini-UART that's bound to the CPU frequency; which is sort-of too bad, because I thought I had something there, for a moment...).

A soft reboot of the pi sometimes makes it happen, and sometimes doesn't. Sometimes "reboot -> garbage" will happen several times in a row.

There is this note in the pymavlink code from @tridge, in ArduPilot/pymavlink@9860922473b2806228453cd3ad47c602d896501a:

        # we rather strangely set the baudrate initially to 1200, then change to the desired
        # baudrate. This works around a kernel bug on some Linux kernels where the baudrate
        # is not set correctly
        self.port = serial.Serial(self.device, 1200, timeout=0,
                                  dsrdtr=False, rtscts=False, xonxoff=False)
        try:
            fd = self.port.fileno()
            set_close_on_exec(fd)
        except Exception:
            fd = None
        self.set_baudrate(self.baud)

I haven't been able to trace down any more information about the bug to which he's alluding. Anyone have a lead on that? More details? "some Linux kernels"?

The thought did occurr to me that we might be experiencing some sort of race with device initialization at bootup, but after adding ridiculously long delays (minutes) before starting mavproxy, I still sometimes saw the mavproxy start up into `garbage mode'.

The quick fix is to just set cron job to periodically flip the baud rate back and forth, e.g.:

stty -F/dev/ttyAMA0 9600; stty -F/dev/ttyAMA0 57600

... but cripes that's a nasty hack.

[rozzin]

... and, no I don't have modemmanager installed.

[QiXuanWang]

I did see noisy output for my trying. But later on when I get a better wire, it disappears. Maybe check the connection? But in my case, restart won't help. So...