Add a badge reader/writer board firmware system

[javawizard]

Per https://makesaltlake.slack.com/archives/CGDLBCCCT/p1610320781275700

We need a way to allow developers to upload new firmware for the badge readers and writers and allow them to download it over-the-air and update themselves whenever it's changed.

Current thinking is to have that baked into Heimdall itself, probably in the form of something like:

• A new model, DeviceFirmware, that has an Active Storage attachment for a binary firmware blob
• Some way of flagging which uploaded firmware should be the current one that gets rolled out to all devices
• Some way of having different sets of firmware for badge readers and badge writers (and other devices in the future)

Open questions:

• Is there any sort of CI pipeline attached to this or is the expectation that a developer builds the firmware on their local computer and uploads it manually to Heimdall?
  • How feasible would a CI pipeline be with Espressif's SDK's requirements?
• How best to support anyone outside of Make Salt Lake who wants to deploy a Heimdall instance? It wouldn't be any good if they had to install Espressif's SDK themselves, build the firmware, and upload it to their own instance. (If CI is feasible, perhaps firmware releases could be built automatically on repo push and Heimdall could pull them in from GitHub and expose them in the UI, with the ability to also upload firmware blobs by hand in case a developer wants to test out some changes they've made locally but haven't pushed.)

[ghost]

The current expectation is that developers would build the firmware on their local machine and upload it manually. But it would be good to have CI to build it automatically. I'm sure it would be possible to do - I'll look into it.

I like the idea of Heimdall pulling firmware from Github and displaying it in a UI. The initial customization will always have to be locally, since without it the firmware doesn't know the WiFi SSID/password: which brings up the point of what to do if (when?) the WiFi password gets changed? Should we be able to build firmware that has the new password embedded in it, and which updates the non-volatile storage key/pair storage area?

[javawizard]

@bcran I totally forgot about the WiFi credentials. That throws a wrench in the works...

Is it possible to update the NVS area programmatically from firmware without otherwise replacing the firmware itself? If so, it seems like another way to do it would be to have firmware bundles that don't embed credentials uploaded to GitHub and then have a separate, out-of-band mechanism for pushing new wireless credentials to the devices, like a separate endpoint in Heimdall, since yeah, that totally needs to be specific to a given Heimdall instance. Would that work? (The initial install would of course still need to be done by hand, but everything after that could be done through the web interface.)

[javawizard]

(Not that any of this needs to be done right off the bat, of course.)

[ghost]

@javawizard Yes, the way it's designed is that the NVS area is always updated separately from the firmware, using the esp-idf nvs tool on the host computer. I presume the firmware running on the device can also update that area itself.

I'm guessing we'd need to support having two sets of WiFi credentials, with the old one being tried until the new one starts working?

[javawizard]

@bcran Yeah, seems like that'd be helpful when doing a switchover. (It's not strictly needed here at the space since our access points can serve multiple SSIDs at the same time, but at least from the Heimdall end it wouldn't be hard to implement.)

So in that case I'll plan to write two different systems, one to expose to badges the WiFi credentials they should use and one to expose the firmware blob they should use.

How do you want to determine if an update is needed? I could write an endpoint to fetch the SHA of the latest firmware bundle - would that be sufficient?

[ghost]

The firmware has a built-in version number, so I think the ESP-IDF OTA API takes care of it.

[javawizard]

Got it. Rephrase: is there anything special you'd like me to do to prevent the firmware from needing to download a full copy of the current firmware version every time it checks for an update? (Or, is the firmware small enough that that's not a problem?)

[ghost]

This may be a little optimistic, but I'm presuming just the first few bytes of the firmware need downloaded to get the firmware description header and check the version.

[ghost]

Otherwise, the firmware .bin file is only 800 KB, so it wouldn't be terrible if it did download it all.