Detect firmware change on OTA based on name & version?

[jpmens]

Homie currently detects a valid OTA request if the version number if receives at ../$ota is higher that the firmware version Homie currently runs.

We're thinking in homie-ota about being able to swap-out firmwares. For example, a device is currently running blink-2.0.1 but it should actually be running temp-1.0.5.

As it stands we could force Homie to OTA by sending it a very high version number (999) to force it to check, and then, when it comes in to actually get the firmware on OTA, we'd slip in temp-1.0.5 to it.

Is there a particular reason why Homie doesn't simply check if the OTA trigger (i.e. the payload of ../$ota) contains a different fwname-fwversion in it?

[jpmens]

This feature could also be used to downgrade a firmware (e.g. if a bug has been introduced -- before being able to fix the bug). And maybe the feature could be non-breaking:

1. If the $ota payload contains, say, a dash (temp-1.0.5) check fwname and fwversion
2. Else proceed as Homie does now

[marvinroger]

A downgrade is possible, as it does not check if the firmware version is higher, but different.

It does not care either if the firmware name is not the same, so you can swap firmware anyway. The only problem is if the loaded firmware is a v1.0.0 and you want to upgrade to b v1.0.0, as the versions are the same

Let's go for the name <separator> version, but I don't like the - separator as - can be present in boh the firmware name and the version. = would not be good either as it is used for the version string in OTA mode, and because the firmware name would be optional, the separator might not exist and cause problems for parsing. What about a @?

[jpmens]

dual-relay@1.0.2 looks good to us!

[jpmens]

The OTA HTTP header currently contains

X-Esp8266-Version = cf3a07e0=h-sensor=1.0.1=1.0.2

with h-sensor being the existing firmware. Do you intend keeping that header as is, or should we expand to

X-Esp8266-Version = <device>=<have_firmware>=<have_version>=<want_firmware>=<want_version>

?

[marvinroger]

No this is what I said when I meant an = as a separator would cause problems for parsing as this is optional. So it would look like:

X-Esp8266-Version = <device>=<have_firmware>=<have_version>=<want_firmware>@<want_version>

And to check if the firmware name is embedded into the wanted firmware, you only have to check if there is an @ in the = splitted string.

[jpmens]

Actually, as Ben found out, it already works a little bit:

X-Esp8266-Version = 0fbbe2e0=dual-relay=1.0.5=dual-relay@1.0.5

But I think this is a bug? 😄

[marvinroger]

This is not a bug, because dual-relay@1.0.5 is different than the version string only. The problem is it will loop, because once flashed 1.0.5 will still be different than dual-relay@1.0.5. If you don't experience this, it means that you did not sent the $ota retained flag to true.

[sumnerboy12]

yep we are not retaining OTA requests by design

[marvinroger]

Alright then, it should work as is for homie-ota. But i'll implement the detection of the @ so that it also works with retained messages.

[jpmens]

Very good, thank you.

[marvinroger]

The swapping of firmware introduces another issue with custom settings (#26). The two firmwares won't have the same custom settings, so the configuration file would have to be erased if another firmware is flashed.

But I don't really see any use case for swapping firmwares. Swapping would make sense if firmwares were only doing software things, in this case it would not be dependent of the physical environment. I mean, each firmware is doing hardware things (controlling a relay, some shutters, a temperature sensor...). @jpmens @sumnerboy12 ?

[jpmens]

I also think swapping a firmware can be associated with a bit more pain because, as you say, that would indeed typically be coupled with some change of hardware.

[sumnerboy12]

Agreed :)

[marvinroger]

And if you change the hardware, you'll need physical access to the ESP8266, so you can flash it "manually". Moreover, you'll want to do it before changing the hardware, as the current loaded firmware might conflict with it (not the same protocol, PINs in a bad state, etc). Not allowing this would be, IMO, safer. Do I understand that we can abandon the idea for the 2.0.0?

[jpmens]

If, by abandon the idea, you mean allow firmware swapping, I say yes. :)

[marvinroger]

Wait, "abandon the idea of allowing firmware swapping", or just "allow firmware swapping"? In other words: firmware swapping, yes or no? :sweat_smile:

[jpmens]

/me dumps core. 😆

firmware_swapping = NO
firmware_updates_over_mqtt = YES

[marvinroger]

Awesome then! :laughing: For firmware over MQTT, here is how it works (already implemented):

1. The device receives an OTA notification from the MQTT broker, as defined in the Homie convention.
2. If the version sent by the broker is different than the one set with Homie.setFirmware(), and if OTA is enabled in the configuration, the device will send a non-retained message to the $ota/request endpoint, asking the entity responsible for OTA to start sending the payload.
3. The entity responsible for OTA then sends the firmware binary to $ota/payload as non-retained, and the device will flash it.
4. Once done, the device reboots as soon as the device is resettable (with Homie.setResettable()).

Is it fine for both of you?

[sumnerboy12]

sounds grand

[jpmens]

I confess it sounds a little brittle because it tries to do RPC over MQTT. :-)

Would this not be easier and possibly more reliable?

1. Broker publishes a retained payload to $ota/payload containing a firmware binary. This can be done at any point in time.
2. Device receives a non-retained OTA notification from broker as defined in the Homie convention
3. If the version sent by the broker in 2 is different and if OTA is enabled, the device will flash the payload it may obtain from 1.
4. Optionally, broker can clear $ota/payload

[marvinroger]

That's what I thought at first, but I told myself it was not doable as you can't select when you want to receive the payload. But the solution is simple: we only have to subscribe to $ota/payload when we receive the notification from the broker. Great :+1:

[marvinroger]

Thinking about it, this might not be a good idea : if you have 10 devices, even if they share the same firmware, your MQTT broker would end up storing 300kB * 10 = 3MB. Whereas, with the request-like method, the broker does not have to store anything.

This also mean that if you have 10 devices, the entity responsible for OTA must send 3MB of data before publishing the $ota notification to each device. With the people using remote broker, like CloudMQTT, I am not even sure the broker will store that much.

[marvinroger]

My bad, you said a non-retained $ota notification... So the OTA entity would monitor when a device connects. It sounds fine!

[jpmens]

Apart from which: 3MB? What's that in today's Giga/Peta/Teraclouds? :-) Honestly, that's not a problem.

And as to using a public cloud for home automation, that to me is a 104.32% NO, quite apart from not having TLS ... ;-)

[marvinroger]

So:

1. The entity responsible for OTA checks the device $fwversion property. If it detects there is a newer version available and $ota/enabled is true, it sends the payload to the device $ota/payload as a retained message.
2. The entity responsible for OTA sends a non-retained notification to the device $ota, with the newer version as the content (act as a guard).
3. The device then subscribes to $ota/payload, actually flashing the new firmware.
4. Once done, the device reboots as soon as the device is resettable (with Homie.setResettable()).
5. The broker, receiving the newer $fwversion, can optionally clean the payload sent to $ota/payload.

So there is no storage problems anymore, as the broker can clean the payload. All good?

[jpmens]

C'est parfait! Make it happen. 😄

[marvinroger]

Done bf330b55d09fcf1159a4c618ecdcb71f6c0cd62c! May I close?

[jpmens]

Closing