search

PX4 / PX4-Autopilot

PX4 Autopilot Software
https://px4.io
BSD 3-Clause "New" or "Revised" License
8.2k stars 13.37k forks source link

USB connection is only accessible if connected in a window of few seconds #15667

Closed AlexisTM closed 3 years ago

AlexisTM commented 4 years ago

Describe the bug

If we do not connect the USB to a consumer right after the bootloader, once the RGB LED starts breathing blue (in a window of few seconds), the Pixhawk is inaccessible, no message can be received or sent anymore, but the TTY ACM driver is still up and running, thinking it can communicate.

I could replicate the exact same behavior going one level down, using libusb1.0 instead of using the kernel usb-cdc-acm driver but the exact same behavior happened. This seems to indicate that the last burst would come from the Pixhawk's buffer and not the tty driver buffer. As the log shows below, if nobody subscribes to the data in /dev/ttyACM0, the USB connection is not initialized.

To Reproduce Steps to reproduce the behavior:

  1. Power the Pixhawk with the USB
  2. Wait for the LED to be blue breathing
  3. Wait an additional 10 seconds
  4. Try to connect Mavros, QGc or cat /dev/ttyACM0
  5. When connecting, few messages are received
  6. The connection hangs: No message can be sent or received

Steps to reproduce the behavior after successful connection:

  1. Power the Pixhawk with the USB
  2. Wait for the LED to be blue breathing
  3. ~Wait an additional 10 seconds~
  4. Try to connect Mavros, QGc or cat /dev/ttyACM0
  5. The connection is successful and can be used for any time you want
  6. Disconnect (stop the node)
  7. Wait 10 seconds
  8. Try to reconnect with the same method
  9. When connecting, few messages are received
  10. The connection hangs: No message can be sent or received

Expected behavior

Steps to reproduce the behavior after successful connection:

  1. Power the Pixhawk with the USB
  2. Wait for the LED to be blue breathing
  3. You can wait any time you want before connecting
  4. Try to connect Mavros, QGc or cat /dev/ttyACM0
  5. The connection is successful and can be used for any time you want

Log Files and Screenshots

sudo apt install vusb-analyzer usbmon

Following a USB log file showing the problem. The Pixhawk in this log is DEV 1018. To view it, download it, rename it to remove the .txt and open it with vusb-analyzer:

usbmon.mon.txt

mv usbmon.mon.txt usbmon.mon # .mon files are not supported on Github
vusb-analyzer usbmon.mon

History of the log:

Normal start: image

Failing start: image

Example log https://review.px4.io/plot_app?log=85f3ab21-074b-4a19-9baa-820d606824c2

To reproduce the log:

sudo su
modprobe usbmon
cat /sys/kernel/debug/usb/usbmon/1u > usbmon.mon

Drone:

dagar commented 4 years ago

Nothing immediately comes to mind here, but I'd suggest you try to instrument the mavlink module and debug through the serial console (or debugger) to see the USB mavlink instance is doing.

dagar commented 4 years ago

To Reproduce Steps to reproduce the behavior:

  1. Power the Pixhawk with the USB
  2. Wait for the LED to be blue breathing
  3. Wait an additional 10 seconds
  4. Try to connect Mavros, QGc or cat /dev/ttyACM0
  5. When connecting, few messages are received
  6. The connection hangs: No message can be sent or received

In step 5 do you happen to know specifically which messages are getting through?

AlexisTM commented 4 years ago

The messages are the typical messages the Pixhawk was supposed to send such as IMU data, position data, ...

Note that I do not have a serial cable as of now and cannot debug further. :disappointed:

Could it be caused by the age of the Pixhawk? It is an old 3DR Pixhawk.

dagar commented 4 years ago

Could it be caused by the age of the Pixhawk? It is an old 3DR Pixhawk.

Unlikely unless the physical connection is questionable.

AlexisTM commented 4 years ago

Using the same cables with other USB-CDC devices, I do not have the issue. (tested with 3 other devices using CDC)

dagar commented 3 years ago

Possible solution - https://github.com/PX4/PX4-Autopilot/pull/16180.

AlexisTM commented 3 years ago

@dagar I will test this! Thanks!

AlexisTM commented 3 years ago

[Closing this in the meantime]