How about OTA updater? :)

[h-david-a]

There are new features almost every day! I'd like to get them directly to the device :) I've tried to adapt micropython_ota: https://github.com/mattjackets/micropython_ota/blob/main/micropython_ota.py But it fails to download 'big' files: in my case it was failing on memory alocation for 44kb Googled a bit, and it appears that there's a mrequests lib providing chunked downloads (https://github.com/SpotlightKid/mrequests/tree/master)

Looking into it. Any inputs are welcome!

[echo-lalia]

I like the idea, but I'm not sure I can envision how this should ideally work for MicroHydra.

Most people are running the compiled firmware, I think. That means that there would need to be a totally separate solution for OTA updates on the firmware, vs running it on normal MicroPython firmware.

Maybe if there was a built-in app that made it easy to download files from GitHub, that could be a good solution? That way, people could fetch new apps from the app repo without needing a computer, and people who want the newest MH updates could download from this repo.

I'm not sure! I'd love to hear your input on this :)

[h-david-a]

So, my perfect setup looks like this:

• I've got M5 stick launcher to select different firmwares if needed (https://github.com/bmorcelli/M5Stick-Launcher/tree/main?tab=readme-ov-file)
• Firmwares stored on SD card
• Currently running micropython clean rom.
• MicroHydra installed from sources (via Thonny) to tinker around. Fun fact, that reflashing (e.g to Bruce or demo roms) doesn't affect MH files (Would be nice if someone explained that to me :))

As for OTA, I see three use cases:

1. For tinkering with a non-compiled version of MH - obviously github downloader is needed. I've got some trials in my repo:

   • adjusted 'original' ota script (it provides app store-like functionality, BUT fails on big files of MicroHydra itself): https://github.com/h-david-a/Cardputer-MicroHydra/blob/draft/silly-tamas/MicroHydra/apps/ota.py
   • non-working 'mrequests' version (which promises to do chunked download to avoid memory issues, but fails with some weird ssl.h exception when trying to save files): https://github.com/h-david-a/Cardputer-MicroHydra/tree/feature/ota_update

2. For updating 'extra' apps of the compiled version - github downloader with store-like functionality is needed (see previous point)

3. For an OTA update of the compiled ROM - some collaboration with M5Stick-Launcher repo needed.

Sidenote for perfectionists: to do OTA sustainably, proper release process is needed for MicroHydra repo (but that's another story once OTA will work at least somehow)

[echo-lalia]

That's a good point; Now that M5Launcher supports MicroHydra, (and that they're adding OTA to it), it basically handles all of the 'OTA' functionality for the compiled version for us.

Fun fact, that reflashing (e.g to Bruce or demo roms) doesn't affect MH files (Would be nice if someone explained that to me :))

I think this happens because MicroPythons virtual filesystem is just not being overwritten by the other firmwares.
Afaik, the normal MicroPython firmware is only taking up a ~2mb partition on it's own, and then the rest of the flash becomes a filesystem. Then, if you overwrite with another firmware, as long as the firmware isn't huge in size, it'll probably only overwrite the MicroPython partition, and not overwrite the filesystem.

You can actually encounter some issues if you, just as an example, install UIFlow, and then install MicroHydra (without erasing the flash first) because the old UIFlow filesystem will be there instead of MH's filesystem.

[echo-lalia]

adjusted 'original' ota script (it provides app store-like functionality, BUT fails on big files of MicroHydra itself):

Would you be willing to share what line of code in your first attempt is running out of memory? I haven't had the chance to try it out myself, but I have encountered a lot of memory issues throughout this project, so I'm curious to know if this will be a familiar issue for me :)

[h-david-a]

That's a good point; Now that M5Launcher supports MicroHydra, (and that they're adding OTA to it), it basically handles all of the 'OTA' functionality for the compiled version for us.

Are they adding OTA?.. Where?

I think this happens because MicroPythons virtual filesystem is just not being overwritten by the other firmwares.

Btw, do you know by chance if there's an 'empty' firmware available? Just to make a clean install? (or that should be a feature request to M5 launcher?)

Would you be willing to share what line of code in your first attempt is running out of memory?

Sure! This is the line (+-1) https://github.com/h-david-a/Cardputer-MicroHydra/blob/67de65a3552fd6d01d7c3c7a6865b7f9deff9506/MicroHydra/apps/ota.py#L105

[echo-lalia]

Pirata, the author of M5launcher, has been sharing updates on the unofficial Cardputer Discord about progress on a launcher that can install from SD and install from the M5 burner repo

Really exciting stuff! They're making progress quickly 😁

And as for erasing the old filesystem... Usually that is something done by the flashing tool you're using, so it makes sense to me that M5 launcher would get that functionality.

[echo-lalia]

Sure! This is the line (+-1) https://github.com/h-david-a/Cardputer-MicroHydra/blob/67de65a3552fd6d01d7c3c7a6865b7f9deff9506/MicroHydra/apps/ota.py#L105

Interesting. So, it looks like it's not failing to fetch the content from the internet, but instead failing to read the content/create the file, right?

If that's the case, you might be able to solve it by doing something similar to what the Files app does. Here's a snippet from the 'paste' function in the files app:

with open(source,"rb") as old_file:
    with open(dest, "wb") as new_file:
        while True:
            l = old_file.read(512)
            if not l: break
            new_file.write(l)

It's not actually reading/writing the file all at once, but scanning through in small chunks, in order to not use too much memory.

I still haven't actually gotten the chance to run your code myself, so I could be totally off with this. But I think it might just be failing because it's trying to fully open/decode the entire file all at once, before writing it to a new file.

[h-david-a]

Thanks! I'll give it a try!

On Wed, 24 Apr 2024, 02:08 Ethan, @.***> wrote:

Sure! This is the line (+-1) https://github.com/h-david-a/Cardputer-MicroHydra/blob/67de65a3552fd6d01d7c3c7a6865b7f9deff9506/MicroHydra/apps/ota.py#L105

Interesting. So, it looks like it's not failing to fetch the content from the internet, but instead failing to read the content/create the file, right?

If that's the case, you might be able to solve it by doing something similar to what the Files app does. Here's a snippet from the 'paste' function in the files app:

with open(source,"rb") as old_file: with open(dest, "wb") as new_file: while True: l = old_file.read(512) if not l: break new_file.write(l)

It's not actually reading/writing the file all at once, but scanning through in small chunks, in order to not use too much memory.

I still haven't actually gotten the chance to run your code myself, so I could be totally off with this. But I think it might just be failing because it's trying to fully open/decode the entire file all at once, before writing it to a new file.

— Reply to this email directly, view it on GitHub https://github.com/echo-lalia/Cardputer-MicroHydra/issues/55#issuecomment-2073687543, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACAY6QCUIG6LC75ZGGASNV3Y63ZZLAVCNFSM6AAAAABGRVILGWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZTGY4DONJUGM . You are receiving this because you authored the thread.Message ID: @.***>

[h-david-a]

So, the issue is with the urequests module. And some nice person already patched that 6 years ago: https://github.com/chrisb2/micropython-lib/blob/master/urequests/example_iter.py :))

[h-david-a]

After messing around got to the No free space on device. Wiping firmware doesn't really help. I was checking available space with shutil (print(shutil.disk_usage('/'))), got total space 1048576 bytes. And current microhydra is +-450KB. OTA downloads +-450KB more. And there are some logs and other leftovers which causes "system crash".

Also, s3 stamp specs mention 8MB flash (http://docs.m5stack.com/en/core/StampS3)

Checking partitions with esp32 method shows also strange results:

print(esp32.Partition.find())

[<Partition type=0, subtype=32, address=65536, size=851968, label=app0, encrypted=0>, <Partition type=0, subtype=16, address=983040, size=5242880, label=app1, encrypted=0>]

So, app0 is 832KB and app1 - 5MB. And that doesn't match any schema described by m5 launcher https://github.com/bmorcelli/M5Stick-Launcher/wiki/Explaining-the-project#custom-partitioning

OTA appears to be a more complicated think :)

[echo-lalia]

FYI, newest version of MicroHydra now has a built-in way to download apps from the github repo. This isn't an OTA updater at all because it can't download/overwrite MicroHydra's files, but I could imagine a future OTA update functionality reusing some of the same logic 😁