search

QubesOS / qubes-issues

The Qubes OS Project issue tracker
https://www.qubes-os.org/doc/issue-tracking/
539 stars 48 forks source link

Android MTP attach fails #6330

Open Eric678 opened 3 years ago

Eric678 commented 3 years ago

Qubes OS version:

R4.0 up to date current, Android 11 (Pixel phone up to date 5 dec 20 - reference platform) f32 target Qube, f32-min sys-usb

Affected component(s) or functionality:

USB handling - dom0

Steps to reproduce the behavior:

Attach Android 11 device with file transfer enabled to a Qube running Nautilus.

Expected or desired behavior:

Entry for device appears in menu in Nautilus and I can select that item to browse the filesystem on the Android device.

Actual behavior:

Menu item appears as expected, however selection goes away for 5-10 sec then gives a popup "Unable to connect to MTP device 00n,00n" and deletes the Nautilus menu item.

General notes:

Please see forum thread for more details: https://qubes-os.discourse.group/t/how-can-i-connect-to-android-via-usb/2101

An older v8 Android does partially work: same error popup in Nautilus, dismiss that, detach and reattach, then works as expected.

If looking into this I did notice another bug in the device widget communication - if Android file transfer is enabled or disabled while attached to a Qube the attachment is dropped in dom0, however the widget is not notified. Attempting detach in the widget gives a not attached error popup.

I have consulted the following relevant documentation:

I am aware of the following related, non-duplicate issues:

0spinboson commented 3 years ago

This also happens here, in my Qubes 4.1 installation.

may or may not be related, but my guest-VM.logs in VMs after I've attacked any type of USB device are being spammed with "usb usb1: Not yet implemented", "vhci_hcd: urb'>status - 104" and "...unlink'->seqnum 7243041" Attaching regular USB devices like microphones or webcams works fine, aside from the log spam.

AlxHnr commented 3 years ago

I have the same issue, but the phone can be mounted from inside sys-usb. (Or by re-attaching the USB PCI device to the VM which is running nautilus). The same applies to adb. See #5717

0spinboson commented 3 years ago

ah yes, it indeed works fine within sys-usb itself.

@marmarek : should I make a separate issue for the log spam issue?

0spinboson commented 3 years ago

btw, the journal in sys-usb also has a stream of entries saying "unlinked by a call to usb_unlink_urb()', repeated every 5s.

mfc commented 3 years ago

i just want to confirm this issue, and that it is also present for target qube and sys-usb based on debian 10.

DemiMarie commented 3 years ago

More generally, it appears that kernel-mode USB drivers work, but user-mode USB drivers fail. Any idea why that is?

iacore commented 2 years ago

Same issue here. lsusb show the device, but cannot mount the device with mtpfs, gvfs-mtp or aft-mtp-cli.

Most verbose log I found:

user@archlinux ~> aft-mtp-cli -v
creating device descriptor at /sys/bus/usb/devices/1-1
probing device 18d1:4ee1
capabilities = 0x000001ff
page size = 4096
<zero-packet> 
<bulk-continuation> 
<no-packet-size-limit> 
<bulk-scatter-gather> 
<reap-after-disconnect> 
<mmap> 
<drop-privileges> 
<conninfo-ex> 
<suspend> 
configurations: 1
interfaces: 1
Device usb interface: 0:0, index: 0, endpoints: 3
read control 80 06 0300 0000
languages[4]:
00000000: 04 03 09 04                                     ....

read control 80 06 03ee 0409
winusb handshake failed: ioctl: Broken pipe
descriptor[57]:
00000000: 12 01 00 02 00 00 00 40 d1 18 e1 4e 40 04 01 02 .......@...N@...
00000010: 03 01 09 02 27 00 01 01 04 c0 00 09 04 00 00 03 ....'...........
00000020: 06 01 01 05 07 05 81 02 00 02 00 07 05 01 02 00 ................
00000030: 02 00 07 05 82 03 1c 00 06                      .........

read control 80 06 0305 0409
interface name[8]:
00000000: 08 03 4d 00 54 00 50 00                         ..M.T.P.

10009 ms since the last poll call
Device::Find failed:timeout reaping usb urb
gw0 commented 2 years ago

Same issue in Qubes 4.1 with USB PCI device in sys-net and Android 11. My hypothesis is the issue appears because the USB is controlled by the Android device (not the PC).

Steps to reproduce and some high-level logs:

## connect USB cable
## on Android set USB controlled by "This device" and use USB for "File transfer/Android Auto"

## dom0:
$ qvm-usb list
BACKEND:DEVID    DESCRIPTION
sys-net:2-1    Google_Pixel_3a_02CA...
$ qvm-usb attach --verbose personal sys-net:2-1

## sys-net:
$ sudo dmesg
[18566.214456] usb 2-1: new high-speed USB device number 36 using xhci_hcd
[18566.345080] usb 2-1: New USB device found, idVendor=18d1, idProduct=4ee1, bcdDevice= 4.40
[18566.345203] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[18566.345302] usb 2-1: Product: Pixel 3a
[18566.345354] usb 2-1: Manufacturer: Google
[18566.345549] usb 2-1: SerialNumber: 02CA...
[18812.887424] usbip-host 2-1: usbip-host: register new device (bus 2 dev 36)
[18812.891413] usbip-host 2-1: stub up

## personal:
$ sudo dmesg
[17256.651442] vhci_hcd vhci_hcd.0: pdev(0) rhport(0) sockfd(0)
[17256.651464] vhci_hcd vhci_hcd.0: devid(131108) speed(3) speed_str(high-speed)
[17256.651493] vhci_hcd vhci_hcd.0: Device attached
[17256.872744] usb 1-1: new high-speed USB device number 14 using vhci_hcd
[17256.990759] usb 1-1: SetAddress Request (14) to port 0
[17257.019751] usb 1-1: New USB device found, idVendor=18d1, idProduct=4ee1, bcdDevice= 4.40
[17257.019836] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[17257.019911] usb 1-1: Product: Pixel 3a
[17257.019949] usb 1-1: Manufacturer: Google
[17257.019986] usb 1-1: SerialNumber: 02CA...
$ jmtpfs ~/mnt
Device 0 (VID=18d1 and PID=4ee1) is a Google Inc Nexus/Pixel (MTP).
PTP_ERROR_IO: failed to open session, trying again after resetting USB interface
LIBMTP libusb: Attempt to reset device
LIBMTP PANIC: failed to open session on second attempt
terminate called after throwing an instance of 'MtpErrorCantOpenDevice'
  what():  Can't open device
Aborted
$ sudo dmesg
[17449.548755] usb 1-1: reset high-speed USB device number 14 using vhci_hcd
[17449.668768] usb 1-1: SetAddress Request (14) to port 0
[17449.812594] usb 1-1: reset high-speed USB device number 14 using vhci_hcd
[17449.931749] usb 1-1: SetAddress Request (14) to port 0
[17509.958094] vhci_hcd: unlink->seqnum 257
[17509.958148] vhci_hcd: urb->status -104

## sys-net:
$ sudo dmesg
[19005.674190] usbip-host 2-1: unlinked by a call to usb_unlink_urb()
[19066.197678] usbip-host 2-1: unlinked by a call to usb_unlink_urb()
[19125.811047] usbip-host 2-1: unlinked by a call to usb_unlink_urb()

## personal:
$ go-mtpfs ~/mnt
2021/12/29 12:33:21 OpenSession failed: LIBUSB_ERROR_TIMEOUT; attempting reset
2021/12/29 12:33:28 Configure failed: OpenSession after reset: LIBUSB_ERROR_TIMEOUT
$ mtp-detect 
libmtp version: 1.1.16

Listing raw device(s)
Device 0 (VID=18d1 and PID=4ee1) is a Google Inc Nexus/Pixel (MTP).
   Found 1 device(s):
   Google Inc: Nexus/Pixel (MTP) (18d1:4ee1) @ bus 1, dev 14
Attempting to connect device(s)
PTP_ERROR_IO: failed to open session, trying again after resetting USB interface
LIBMTP libusb: Attempt to reset device
LIBMTP PANIC: failed to open session on second attempt
Unable to open raw device 0
OK.
iacore commented 2 years ago

My hypothesis is the issue appears because the USB is controlled by the Android device (not the PC).

USB tethering works (if connecting Android device to sys-net) and that is controlled by Android.

gw0 commented 2 years ago

My hypothesis is the issue appears because the USB is controlled by the Android device (not the PC).

USB tethering works (if connecting Android device to sys-net) and that is controlled by Android.

Sure, everything works in sys-net, because the USB PCI device is attached to this VM (as @AlxHnr said). For example MTP attach in sys-net when controlled by Android:

## sys-net:
$ sudo apt-get install jmtpfs
$ jmtpfs mnt
Device 0 (VID=18d1 and PID=4ee1) is a Google Inc Nexus/Pixel (MTP).
Android device detected, assigning default bug flags
$ ls -al mnt
total 4
drwxr-xr-x  3 user user    0 Jan  1  1970  .
drwxrwxrwt 14 root root 4096 Jan  3 09:50  ..
drwxr-xr-x 22 user user    0 Nov 25  8502 'Internal shared storage'
$ fusermount -u mnt

But the forwarding/reattaching just the Android USB device (not PCI) to another VM does not work. Because this works perfectly for slave USB devices, my hypothesis is that the issue is because USB is controlled by the Android device.

iacore commented 2 years ago

I meant sys-net, not sys-usb.

In USB tethering mode, it's still controlled by the Android device. Yet it works (connecting the android device form sys-usb to sys-net.

AlxHnr commented 2 years ago

Still occurs on the latest Qubes 4.1 with fedora 34 as App VM.

Szewcson commented 2 years ago

In android 12 there is option to change the device that controlling usb. Unfortunately it causes detachment from VM, and ends with error on android side. I tried with attaching android device persistently but it not works. It also not work on Windows VM as I mentioned in #7731

SurienDG commented 2 years ago

Yea I think the main problem is the detachment when changing "Use USB for" options on your Android device. In Android 8 in order for it to work for me I had to set my device to default to always connect with mtp mode rather than the charging mode (since it couldn't be changed once connected to a VM). Now in Android 13 when I first connect it in sys-usb with the "No data transfer" option and I click to mount it works but gives me an empty folder with no files (since I selected no transfer). However, once I disconnect it and try to reconnect it (still in sys-usb) I get the "Unable to connect to MTP device 00n,00n" error.

Now if I change the mode on my Android device to "File transfer" and try to mount it again then it works successfully in sys-usb. Now if I unmount it after doing this and try to reconnect still in sys-usb still with the "File transfer" option I again get the MTP error.

Thus, this makes me think that the issue is that on Android 11+ right after you set your option for "Use USB for" you can mount but cannot remount without changing your setting which could explain why it cannot be assigned from sys-usb to another qube because that would result in another mount after the first successful one in sys-usb.

Does that make sense? If is there a way we can prevent consecutive mounts as that seems to be causing the problem on newer Android versions based on my testing and what I've read in this issue thread. What do you all think?

Szewcson commented 2 years ago

I was finally able to connect using windows VM and images transfer mode on android. But it works only sometimes and I was not able to figure out during which conditions. File transfer mode never work for me, and usb modem mode seems to work almost always.

UndeadDevel commented 10 months ago

Affects 4.2.

FWIW, in sys-usb the Android phone does automount and can immediately be opened and browsed in Thunar, but attaching it to any other qube that I've tried doesn't seem to work...no errors in dom0 dmesg or journalctl but nothing new pops up in /dev/ or lsblk in the target qube.

DemiMarie commented 10 months ago

Nothing in lsblk is expected. MTP does not expose a block device and is implemented in userspace.

UndeadDevel commented 10 months ago

Nothing in lsblk is expected. MTP does not expose a block device and is implemented in userspace.

Ah. Well, forgive my ignorance; it does show up in the Devices widget under "USB Devices", however, so I thought it should at least pop up as a device in /dev/, but nothing new appears there.

I've tried jmtpfs mymountpoint but that fails with

Device 0 (VID=0e8d and PID=2008) is a MediaTek Inc MT65xx.
PTP_ERROR_IO: failed to open session, trying again after resetting USB interface
LIBMTP libusb: Attempt to reset device
LIBMTP PANIC: failed to open session on second attempt
terminate called after throwing an instance of 'MtpErrorCantOpenDevice'
  what():  Can't open device
Aborted
DemiMarie commented 10 months ago

@UndeadDevel did you remember to attach the USB device?

UndeadDevel commented 10 months ago

Of course...using the Devices widget.

If I turn File Transfer off on the phone after attaching and then detach (again using the widget) then it gives an error saying that nothing was attached, so I presume it detaches itself in that case; if I leave File Transfer enabled, however, there is no error, so I assume that it is attached and detached as expected when I trigger these events using the widget.

marmarek commented 10 months ago

AFAIR Android resets the USB device side when switching mode to MTP (or any other). This detaches the device from the target qube, but it seems to be possible to select file transfer mode before attaching the device, and the selected mode seems to survive attaching to a qube. And also, while I get exactly the same error, the "libusb: Attempt to reset device" seems to not detach device from the target qube anymore, which is a progress I'd say.

At this point, I can confirm it doesn't work, but also I don't see any obvious reason why. The reset detaching the device (or attach resetting the MTP mode) used to be a problem, but it isn't the case anymore. "USB tethering" mode seems to work just fine, so it isn't generic Android USB issue, it is something specific to MTP.

DemiMarie commented 9 months ago

This detaches the device from the target qube

I’m pretty sure this is the bug. User-mode applications expect that the device will remain attached in this case and break otherwise. There may be a timeout, but it is too short for anything that requires user confirmation. For Android to work, reattachment will need to happen automatically and without a prompt.

Is it possible to distinguish resetting of the USB device from the USB device being physically disconnected?

marmarek commented 9 months ago

Is it possible to distinguish resetting of the USB device from the USB device being physically disconnected?

In general case - no. Some devices even really disconnect data lines on reset; and AFAIR the XHCI interface doesn't give you much more than "disconnect" + "connect" events in that case. But maybe in some cases there is some trick to detect such case.

But also, see my comment above: disconnect on mode switch is no longer the thing that breaks MTP, so this aspect is offtopic for this issue.

DemiMarie commented 8 months ago

I’m guessing that some system call is failing for some reason.

ryrona2 commented 6 months ago

This this issue affects me as well, I decided to do some investigations. I used Wireshark in sys-usb to monitor the /dev/usbmonX device node my Android device is connected to. My Android device runs latest Graphene OS.

I have found the problem. If more than one SET_CONFIGURATION control packet is ever sent to the Android device, it will refuse to answer to any data packet sent to it forever, and only answer to control packets.

This can happen in the following cases:

Case 1: One SET_CONFIGURATION is sent to enable the device when it is attached to the qube running the USB driver, typically sys-usb. When the USB device is assigned in into another qube, two more SET_CONFIGURATION is sent, one to disable and one to re-enable the device, the latter originating from the newly assigned qube rather than the one running the USB driver. The MTP device cannot be mounted from within any qube, all data packets from host to device will timeout without getting any response.

Case 2: One SET_CONFIGURATION is sent to enable the device when it is attached to the qube running the USB driver. Now, the MTP device can be mounted and browsed just fine from within that qube. But when unmounting the MTP device, a PORT_RESET/C_PORT_RESET is sent to the device and then after that of course a new SET_CONFIGURATION is sent to enable the device after the reset. Now it won't work to mount the MTP device again, not even from the qube running the USB driver. All packets from host to device will timeout without getting any response.

Case 3: One SET_CONFIGURATION is sent to enable the device when it is attached to the qube running the USB driver. You can manually send another to "enable" the device again despite it already being enabled by "echo 1 > bConfigurationValue" for the /sys node of the USB device. Now mounting the MTP device even from within the USB driver qube won't work. All packets from host to device will timeout without getting any response.

This only affects Android devices. The Android device can easily be reset to work again by either unplugging and replugging the cable, or by switching USB mode to "No data transfer" and then back to "MTP" in the UI of the Android device. Resetting the device from host does not work however, it must be reset physically or from the Android device.

All parts of Android USB interfacing components seem to have the same or similar issues, including fastboot (the bootloader interface). USB debugging (adb) is a bit different, because it actually resets the device if it gets a second SET_CONFIGURATION, and then recovers cleanly. But this still prevents assigning it into another qube, since resetting the device means it momentarily disappears as if unplugged.

This issue can also be reproduced on Ubuntu using case 2 and 3, so the issue is not Qubes OS specific.

All other devices than Android devices seem to handle multiple SET_CONFIGURATION just fine, including disabling an re-enabling. But Android seem to have issues in most or all of its USB facing components.

My current suspicion is a driver bug in Android (and Pixel bootloader for fastboot), or possibly that some hardening feature against USB tampering on Android (and Pixel bootloader) is triggering on this, distrusting host until user resets the connection manually. I do not believe Qubes OS nor Ubuntu is at fault here.

Logical next step in debugging this would be to try figuring out exactly why Android decides to no longer respond, but this involves debugging on the Android device. A bug report against Android should probably also be opened. I am uncertain if Qubes OS should attempt any workaround, or what such a workaround could be.

DemiMarie commented 6 months ago

Nice catch @ryrona2!

DemiMarie commented 6 months ago

The obvious workaround is for sys-usb to not send SET_CONFIGURATION at all. @ryrona2: can you test the following setup?

If you run into #9104, the patch in QubesOS/qubes-app-linux-usb-proxy#35 will make attach succeed again.

My hope is that with this setup, sys-usb never sends SET_CONFIGURATION at all.

ryrona2 commented 6 months ago

I did test this quickly. Now sys-usb wouldn't send any SET_CONFIGURATION anymore, but the qube I assign the device into sends multiple, including one to disable the device and one to re-enable it. I don't really understand why it does this, it should have behaved like sys-usb usually does, just enabling the device once. But MTP still didn't work inside the qube probably because of the multiple SET_CONFIGURATION calls.

I am wondering, maybe the best solution really would be an MTP proxy, similar to how block devices are proxied. This way sys-usb would still handle the SET_CONFIGURATION, but only sys-usb, because the actual USB device is not assigned into any qube, just the MTP proxy. Because, I mean, even if we did prevent sys-usb from setting up the device like you suggest, if we re-assign the USB device between qubes, each qube would still send a new SET_CONFIGURATION, so it would still be pretty broken for devices that does not support multiple SET_CONFIGURATION calls.

I had a quick chat with someone on GrapheneOS developer chat, and they didn't seem to excited about changing the behavior, saying Linux and QubesOS is doing wrong if they ever send more than one SET_CONFIGURATION call, because they say neither Windows, MacOS or even Android ever does that. I don't really know what the USB standard says.

DemiMarie commented 6 months ago

I did test this quickly. Now sys-usb wouldn't send any SET_CONFIGURATION anymore, but the qube I assign the device into sends multiple, including one to disable the device and one to re-enable it. I don't really understand why it does this, it should have behaved like sys-usb usually does, just enabling the device once. But MTP still didn't work inside the qube probably because of the multiple SET_CONFIGURATION calls.

Is there a SET_CONFIGURATION after the device is attached to the VM, but before MTP tries to use the device? I think this is a USBIP bug.

I am wondering, maybe the best solution really would be an MTP proxy, similar to how block devices are proxied. This way sys-usb would still handle the SET_CONFIGURATION, but only sys-usb, because the actual USB device is not assigned into any qube, just the MTP proxy. Because, I mean, even if we did prevent sys-usb from setting up the device like you suggest, if we re-assign the USB device between qubes, each qube would still send a new SET_CONFIGURATION, so it would still be pretty broken for devices that does not support multiple SET_CONFIGURATION calls.

I had a quick chat with someone on GrapheneOS developer chat, and they didn't seem to excited about changing the behavior, saying Linux and QubesOS is doing wrong if they ever send more than one SET_CONFIGURATION call, because they say neither Windows, MacOS or even Android ever does that. I don't really know what the USB standard says.

This seems like a Linux kernel bug to me. Could you report this on linux-usb@vger.kernel.org?