userland bootloader mode

[ladyada]

i was thinking about how the microbit has a separate chip for uploading, so no reset button is required...i've been showing people makecode and even i would sometimes forget to press-reset-button-to-bootload.

my thought is: is there any way we can do to basically run UF2 in usermode from within makecodes' runtime? that is, once they download a make-code project, it will also include a mass-storage background task, when you drag a uf2 to, will upload the code? this will make it a little more like how circuitpython works, where you can just drag/save code whenever you like :)

here's two ways i think it could work... either we could delegate 1/2 of 'top' flash as an untouchable 'OTA' section. you lose half flash but thats still ~128KB available (is that enough? do we need all 256KB?) when a UF2 is copied in during 'user mode', its depacked and stuffed into the top half of FLASH, then it immediately jumps to the UF2 section to a function that runs something like

    for (i = BOOTLOADER_K * 1024; i < USERCODE_MAX_ADDR ; i += FLASH_ROW_SIZE) {
        flash_write_row((uint32_t *)(void *)i, (uint32_t *)(void *)pageBuf);
    }
   rjmp 0x2000 // or whatever the user address is 

i think we can fit that into the bootloader - it would just iterate to flash all the code and jump, either to the user addr or to to 0x0 and let the bootloader manage the reset

or...

we could do the above but instead of jumping to the bootloader code, basically have the UF2 flashing code manually placed at the top of flash, and it just is really careful not to overwrite itself :)

it seems to me the big challenge would be adding the MSD to makecode runtime, and the 1/2 flash loss. any thoughts @pelikhan @mmoskal @tballmsft?

[finneyj]

Hi @ladyada

This would definitely be a better user experience for sure. I'm not quite so sure about writing off 50% of FLASH though - seems a high price to pay...

I was wondering if we could bring up an MSD with the same geometry as UF2 in the runtime, then whn we detect a block write with a UF2 signature, do some cleanup to ensure a sane state then handoff control into the bootloader to continue the download... We'd need to find a way to ensure we can share appropriate connection state though, so that the USB I/F doesn't explode. :-)

[ladyada]

well. there's another option: we have a 2MB SPI flash on board. you could stuff the depacked code in there. getting out requires more work on UF2's part to get it out. not a ton but it may not fit in UF2 as is since some bit-twiddling is required to get the data out of FLASH. (that said, it may be worth trying to fit it in and maybe squish something else)

trying to do a handoff between user and UF2 mid-write may be possible - but i think it would be pretty hairy :/

[finneyj]

oooh, interesting. Using the SPI flash as a buffer is a pretty neat. plan. Setting aside a maximum of 256KB of the 2MB SPI flash (12.5%) seems like a good trade off to me.

With this route we'd likely be better off reflashing directly from the runtime, (but using the UF2 protocol), and leaving the UF2 bootloader as the backup escape route. Also paves the way for partial reflashing (something on the cards for BLE on micro:bit already)...

[tannewt]

Does the makecode section change every flash? Or could we think of the makecode runtime as a second bootloader essentially?

[ladyada]

@finneyj sure, so you're suggesting MC shows up as a drive, they drag the file, on detection of UF2 blocks, MC stuffs them into the SPI flash, takes down all interrupts and USB code, de-enumerates, and jumps to a special writer function. of course, we'd have to copy the writer function to a safe spot, since otherwise it would overwrite it self.

[finneyj]

@ladyada yep - precisely that.

@tannewt The makecode user app is essentially linked with a static image of the runtime, so no need for a full reflash unless unless we update to a new version of the run time, Based on experiences with microbit, the common case of users tweaking their code requires only a pretty small change typically, so there could be performance gains to be had here.

I guess, yes, we could think of it as providing a second level loader...

[mmoskal]

The hand over from user space to bootloader is already supported. I got it working with MP some time ago. Essentially when user space detects a UF2 block being written it calls into bootloader which handles rest of the write. It never returns. It only uses stack. The parameters of the MSD don't matter because I only need to detect uf2 blocks.

[finneyj]

prefect - the ideal way to go.

[ladyada]

@mmoskal neat! sounds like you thought of all this already :)

[ladyada]

(that said - i still think userland USB MSD would be cool because you could use the 2MB flash drive for storing files, javascript sketches, configs, saving datalogged data, or playing/recoding audio files)

[finneyj]

for sure!

[ladyada]

(disappears back to work on hardware)

:)

[finneyj]

(disappears to bed) :-)

[mmoskal]

You need user land MSD for this hand over to work. It makes sense to expose the spi flash through it.

[finneyj]

@mmoskal yes, for sure.

[ladyada]

ok neat :) when y'all have the enumeration working, @tannewt has MIT-licensed, ASF-ready SPI flash management code and also FAT filesystem interface code you can pop in place. same code we use in python so its well tested and DMA optimized!

[ericbaril72]

Is there a Specific branch I should look for if you'd like a hand with testing ? This SPI-flash proposal is a very good tease.

I am not a pxt-adafruit user BUT I do love this UF2 bootloading. Started using UF2 with both ArduinoIDE AND Atmel Visual Studio.

Will most likely bridge UF2 with the WINC1500. Allready did a WINC1500->SDcard-->bootloader but the UF2 method has better Flash-memory handling.

It'd be awesome if the WINC1500 had a bit of USERspace code avail ... could mix OTA with UF2 without a dedicated bridging-code OR loosing half the FLASH space.