macOS 15 Sequoia finder drag&drop flashing fails

[microbit-carlos]

Not quite the same, but similar to:

• https://github.com/ARMmbed/DAPLink/issues/982

There has been reports of issues flashing in macOS 15 Sequoia via finder.

Some things we've noticed so far:

• There is a lot of USB activity when the drive is mounted, 20 to 60 seconds (might depend on computer speed, a colleague with a mac mini sees this for 20 seconds, the slow VM I use for testing takes 60 seconds).
• If a hex file is flashed during the activity period, the flash succeeds
• If a hex file is flashed after this activity is done, the flash fails
  • Most common errors encounter are ERROR_TRANSFER_TIMEOUT and ERROR_OOO_SECTOR
  • This is replicable in the CLI with the cp and rsync commands as well
• WebUSB flashing works correctly

Issue description

• Connect the micro:bit via USB to a macOS 15 computer
• Wait for the orange LED to stop blinking
• Drag & drop a Universal hex file into the micro:bit USB Drive (named "NO NAME" due to https://github.com/ARMmbed/DAPLink/issues/1082)
  • Any file from the micro:bit MakeCode or Python editors will do, for example: universal-hex-micropython.hex.zip
  • A normal Intel Hex file also has this problem, but because the file can be much smaller it might present less obvious symptoms. For example: blink-a-b.hex.zip
• The file transfer starts and the normal Copying "file.hex" to ... window with a progress bar appears
  • While the window is open we can see the orange LED blinking
  • If a Universal Hex is used, the target is never halted, which is the first step before it erases the target flash and starts writing the hex data.
    • When DAPLink processes a Universal Hex it will only halt and erase the target when it reaches the V2 section of the hex, this is usually placed in the second half of the file. So, this can be caused if anything causes DAPLink to stop processing the hex data during the first half
• The time this file transfer window is open is usually shorter than a normal flash
  • Most of the time during a file transfer is spent processing the hex data and flashing the target, if DAPLink starts dropping the hex data blocks, then that normally shortens the file transfer time
• The file transfer window closes and the orange LED continues blinking for a few seconds
• After about 30 seconds the USB drive unmounts and remounts, a FAIL.TXT appears usually with the "timeout" or the "out of order blocks" error
• At this point DAPLink is meant to erase the target and reflash it with a hex file that scrolls the error code on the micro:bit display, this doesn't always happen

Initial findings

I've done an initial test by printing some vfs data to serial during the file transfer. I'll need to do a better capture using RTT next.

With this initial run, we can see that the captured flashing failures are not caused due to out-of-order blocks, but because the OS now sends continuous file blocks to non-continuous fs sectors (the fs is configured with 512 byte sectors).

• Hex file used: uhex-sectors-upy-longrecords.hex.zip
• Serial logs captured: serial_0.txt serial_1.txt

For example, in the serial_0.txt log we can see:

• It starts by doing a lot of read and writes, I don't have info about what these could be, more info should be collected with RTT
• It then starts to send the file data, starting from the beginning of the file to go into sector 357.
  • This is correctly parsed by DAPLink, in V2 this does not trigger the halt+erase yet, because I used a Universal Hex file and the first half of the hex file contains the V1 data
• Then 8 file blocks later, it sends the next file block that should normally go into sector 365 into sector 381 instead
  • DAPLink rejects this block as it was expecting a block for sector 365, and it will continue to reject all the blocks sent by the OS as it is waiting for a block for sector 365
• It does this a few more times, but at least the file blocks appear to be continuous
  • I'm checking this log manually, and haven't checked every single write entry to ensure they are all continuous, but every 100 or so blocks instead, and I haven't noticed any out of order block
• By the time it reaches sector 692, then next block is sent to sector 1677
• There are a few more smaller jumps and a slightly larger one (from sector 1996 to 2157), and then it finally writes the last file block into sector 4758
• There are a few more writes to lower sectors (below the initial sector 357 where the first hex file block was written into) all over the log, which we'll need to look into with RTT. Also a few more writes to sectors in-between the hex file jumps (for example around sector 1220, which is not used for the hex file)

[fabianhugo]

I am seeing something very similar on Ubuntu 22.04: uname -a shows: Linux hugoUbu20 6.8.0-45-generic #45~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Sep 11 15:25:05 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

Copying of universal hex often fails with error 504 and the fail.txt reads:

error: The transfer timed out.
type: transient, user

Around 3 of 4 file transfer fail. Using cp or rsync in the terminal do also not improve the behavior. And WebUSB is not affected as well.

One way for me to tell that the copying failed is to check the contents of the mass storage. If the hex is visible there, it failed.

[martinwork]

@fabianhugo For Linux, this UDEV rules file 99-microbit-udisks.zip can help by adding the "sync" option to MICROBIT drives.

To install...

Connect micro:bit and check current options: mount | grep MICROBIT

Unzip 99-microbit-udisks.zip, and copy the rules file to the udev rules.d folder. For example sudo cp 99-microbit-udisks.rules /etc/udev/rules.d

Check it has the same owner, group and permissions as other rules files. ls -l /etc/udev/rules.d

Restart UDEV. sudo udevadm control --reload-rules ; sudo udevadm trigger

Disconnect and reconnect micro:bit then check what options are in use now. mount | grep MICROBIT

The only difference should be the addition of "sync".

To uninstall, delete 99-microbit-udisks.rules from the rules.d folder.