iPhone 7+/iOS 13.2.2: Timeout reading from or writing to the port

[Algebro7]

Hello,

I'm trying to interact with my iPhone 7+'s camera roll and for some reason libgphoto2 cannot talk to it at all after I plug it in and pair it. gphoto2 --auto-detect shows the device (although it lists it as an iPhone 5 for some reason):

$ gphoto2 --auto-detect
Model                          Port                                            
----------------------------------------------------------
Apple iPhone 5 (PTP mode)      usb:001,005

However, any interaction with the device times out:

$ gphoto2 --list-folders

*** Error ***              
PTP Timeout

*** Error ***              
An error occurred in the io-library ('Timeout reading from or writing to the port'): No error description available
*** Error (-10: 'Timeout reading from or writing to the port') ***

System info: Arch Linux Kernel 5.3.11 (also tried 4.18.84 LTS) gphoto2 2.5.23 libgphoto2 2.5.23 libgphoto2_port 0.12.0

I can mount the filesystem just fine with ifuse and dmesg seems to show the device being picked up correctly and usbmuxd starts automatically, so it seems to me that this is related to the PTP functionality, although at this point I really have no idea.

Let me know if I can provide any other helpful information.

update: I noticed the arch-linux package for usbmuxd is pinned to a certain (old) commit, so installing the full stack from git allows me to access the device again. However, at some point when scanning the images with digikam, Libgphoto2 starts throwing the following error for every image it parses:

Nov 14 13:59:59 arch digikam[2098]: digikam.import: Failed to get camera item!
Nov 14 13:59:59 arch digikam[2098]: digikam.import: Libgphoto2 error:  Unsupported operation  ( -6 )

And later when it actually tries to download them:

Nov 14 14:12:50 arch digikam[2098]: digikam.import: Downloading:  "IMG_2630.MOV"  using  "/home/algebro/Pictures/iphone/2019-06/Camera-tmp1-2098.digikamtempfile.IMG_2630.MOV"
Nov 14 14:13:11 arch digikam[2098]: digikam.import: Failed to get camera item!
Nov 14 14:13:11 arch digikam[2098]: digikam.import: Libgphoto2 error:  I/O problem  ( -7 )

[Mrcrypt]

I have the same problem, but only some pictures cant be downloaded. I could import like 1000 photos, but about 15 of them cant be downloaded. I cant find the issue.

3.676524 save_file_to_file           (2): using fd method
3.676531 gp_camera_file_get          (2): Getting file 'IMG_E8866.JPG' in folder '/store_00010001/DCIM/108APPLE'...
3.676536 gp_filesystem_get_file_impl (2): Getting file 'IMG_E8866.JPG' from folder '/store_00010001/DCIM/108APPLE' (type 1)...
3.676540 lookup_folder_file          (2): Lookup folder /store_00010001/DCIM/108APPLE file IMG_E8866.JPG
3.676544 lookup_folder               (2): Lookup folder '/store_00010001/DCIM/108APPLE'...
3.676555 gp_filesystem_get_file_impl (2): Downloading 'IMG_E8866.JPG' from folder '/store_00010001/DCIM/108APPLE'...
3.676562 ptp                         (2): (storage=0x00010001, handle=0x00000000)
3.676566 ptp                         (2): (storage=0x00010001, handle=0x00000001)
3.676572 ptp                         (2): (storage=0x00010001, handle=0x0000009e)
3.676617 get_file_func               (2): Getting file 'IMG_E8866.JPG'.
3.676629 ptp_usb_sendreq             (2): Sending PTP_OC 0x101b (Get partial object) (0x4d3,0x0,0x100000) request...
3.676633 gp_port_write               (3): Writing 24 = 0x18 bytes to port...
3.676846 gp_port_write               (3): Wrote   24 = 0x18 bytes to port: (hexdump of 24 bytes)
0000  18 00 00 00 01 00 1b 10-e4 04 00 00 d3 04 00 00  ................
0010  00 00 00 00 00 00 10 00-                         ........        

3.676868 ptp_usb_getdata             (2): Reading PTP_OC 0x101b (Get partial object) data...
3.676878 gp_port_read                (3): Reading 1024 = 0x400 bytes from port...
23.680303 gp_libusb1_read [libusb1.c:609](0): 'libusb_bulk_transfer (port->pl->dh, port->settings.usb.inep, (unsigned char*)bytes, si
ze, &curread, port->timeout)' failed: Operation timed out (-7)
23.680356 gp_port_read [gphoto2-port.c:441](0): Reading 1024 = 0x400 bytes from port failed: Timeout reading from or writing to the p
ort (-10)
23.680370 ptp_usb_getdata [usb.c:479] (0): PTP_OC 0x101b receiving data failed: PTP Timeout (0x02fa)
23.680383 get_file_func [library.c:7676](0): 'ptp_getpartialobject (params, oid, offset, xsize, &ximage, &xlen)' failed: 'PTP I/O Err
or' (0x02ff)
23.680396 gp_context_error            (0): PTP I/O Error
23.680436 gp_filesystem_get_file      (2): Download of 'IMG_E8866.JPG' from '/store_00010001/DCIM/108APPLE' (type 1) failed. Reason: 
'I/O problem'
23.680449 gp_camera_file_get [gphoto2-camera.c:1693](0): 'gp_filesystem_get_file (camera->fs, folder, file, type, camera_file, contex
t)' failed: -7
23.681249 gp_camera_free              (2): Freeing camera...
23.681276 gp_camera_exit              (2): Exiting camera ('Apple iPhone 5 (PTP mode)')...
23.681299 ptp_usb_sendreq             (2): Sending PTP_OC 0x1003 (Close session) request...
23.681315 gp_port_write               (3): Writing 12 = 0xc bytes to port...
23.681519 gp_port_write               (3): Wrote   12 = 0xc bytes to port: (hexdump of 12 bytes)
0000  0c 00 00 00 01 00 03 10-e5 04 00 00              ............    

23.681553 ptp_usb_getresp             (2): Reading PTP_OC 0x1003 (Close session) response...
23.681567 gp_port_read                (3): Reading 1024 = 0x400 bytes from port...
23.682645 gp_port_read                (3): Read    12 = 0xc out of 1024 bytes from port: (hexdump of 12 bytes)
0000  0c 00 00 00 03 00 01 20-e5 04 00 00              ....... ....    

23.682787 gp_port_close               (2): Closing port...

[Mrcrypt]

So the problems seems to be that IOS saves some images in the HEIC format and converts them to JPG if we try to recieve them. I think this conversion takes more time than the timeout. I couldnt figure out where the timeout is.

A workaround is as described in here https://support.apple.com/en-us/HT207022 in section "Importing this media via USB" to tap keep originals. Now no conversion happens and we can import the pictures.

[msmeissn]

we can rev the timeout from 20 seconds to something higher. is there an upper limit to the time taken?

[Mrcrypt]

I think the problem occurs before the long read happens. We want to read 1024 bytes and only read 12 bytes because the jpg is converting. Then we try to read again and we get an incomplete transfer code from the phone, because we didnt wait long enough in the first read. Then we try to read again and get a timeout after 20 seconds because no operation is in progress, because it already aborted the transfer.

I commented with "<-----" in the log below. It appears the timout doesnt apply to the line with Here

Working transfer

2.133355 get_file_func               (2): Getting file 'IMG_E8404.JPG'.
2.133364 ptp_usb_sendreq             (2): Sending PTP_OC 0x1009 (Get object) (0x40a) request...
2.133369 gp_port_write               (3): Writing 16 = 0x10 bytes to port...
2.133488 gp_port_write               (3): Wrote   16 = 0x10 bytes to port: (hexdump of 16 bytes)
0000  10 00 00 00 01 00 09 10-df 04 00 00 0a 04 00 00  ................

2.133506 ptp_usb_getdata             (2): Reading PTP_OC 0x1009 (Get object) data...
2.133516 gp_port_read                (3): Reading 1024 = 0x400 bytes from port...
2.198492 gp_port_read                (3): Read    1024 = 0x400 bytes from port: (hexdump of 1024 bytes)
0000  cf e6 02 00 02 00 09 10-df 04 00 00 ff d8 ff e1  ................
0010  01 08 45 78 69 66 00 00-4d 4d 00 2a 00 00 00 08  ..Exif..MM.*....
0020  00 05 01 1a 00 05 00 00-00 01 00 00 00 4a 01 1b  .............J..
0030  00 05 00 00 00 01 00 00-00 52 01 28 00 03 00 00  .........R.(....

Not working transfer

1.852171 get_file_func               (2): Getting file 'IMG_8866.JPG'. 
1.852184 ptp_usb_sendreq             (2): Sending PTP_OC 0x1009 (Get object) (0x4d1) request...
1.852189 gp_port_write               (3): Writing 16 = 0x10 bytes to port...
1.852436 gp_port_write               (3): Wrote   16 = 0x10 bytes to port: (hexdump of 16 bytes)
0000  10 00 00 00 01 00 09 10-df 04 00 00 d1 04 00 00  ................

1.852452 ptp_usb_getdata             (2): Reading PTP_OC 0x1009 (Get object) data...
1.852460 gp_port_read                (3): Reading 1024 = 0x400 bytes from port...
2.024140 gp_port_read                (3): Read    12 = 0xc out of 1024 bytes from port: (hexdump of 12 bytes)
0000  a5 ef 0a 00 02 00 09 10-df 04 00 00              ............    <---------**HERE** we have to read longer I think

2.024242 gp_port_read                (3): Reading 524288 = 0x80000 bytes from port... <--- Dont try to read again we have to wait longer in the read before
2.024589 gp_port_read                (3): Read    12 = 0xc out of 524288 bytes from port: (hexdump of 12 bytes)
0000  0c 00 00 00 03 00 **07 20**-df 04 00 00              ....... ....    <------- We get Incomplete transfer code 0x2007

2.024644 gp_port_read                (3): Reading 524288 = 0x80000 bytes from port...
22.028232 gp_libusb1_read [libusb1.c:607](0): 'libusb_bulk_transfer (port->pl->dh, port->settings.usb.inep, (unsigned char*)bytes, si
ze, &curread, port->timeout)' failed: Operation timed out (-7)

[msmeissn]

This opcode / reply sequence is not something our stack currently expects. If the device announces this large blob of data to transfer, we expect it to transfer....

This 0x2007 error is also not caused by our timeouts here, but by the remote device.

[Mrcrypt]

I think the workaround is sufficient, maybe its worth a note in a readme. If Apple chooses to remove the option in the future it may be worth revisiting the issue. Currently I dont think its justifying bigger changes to resolve this weirdness from Apple.

[krzyk]

This workaround doesn't always work, e.g. it doesn't on my new iPhone 12 mini :(

[Cyp]

The workaround seems to have worked for some of the .JPG and .MOV files, but then it failed on other .JPG and .MOV later on (but same error as for the files that were failing pre-workaround).

Would be good to have a way of skipping failed files as a last-resort option… Using --debug to see which file failed, then doing touch FAILEDFILE.JPG so that it skips it on the next attempt isn't very practical.