OTA firmware update

[jmarcelino]

I was trying out the over-the-air (OTA) firmware update functionality and after a small tweak was surprised I could actually:

1. Build a new image
2. Upload the newly built appimg.bin to /flash/sys/appimg.bin using FTP
3. New OTA image was picked up and everything worked on reboot!

The only change I made was setting

define IMG_SIZE (1536 * 1024)

in pycom-micropython-sigfox/esp32/bootloader/bootloader.h

(the 1536K is the actual size of the image, the old value 1024 overwrote the running code on SPI and cause very bad results)

I understand the size of the images and maybe the bootloader may not be fixed in stone yet but maybe this could work within minor versions?

[Mungio]

I'm sorry for the lack of experience, but which tool of this repository I must use to create the appimg.bin?

Thx.

[jmarcelino]

@Mungio You need to setup the build environment first, see the "The ESP32 version" section in this repo's https://github.com/pycom/pycom-micropython-sigfox README up to the "To build and flash" instructions

Then follow the README in the esp32 folder https://github.com/pycom/pycom-micropython-sigfox/tree/master/esp32

The appimg.bin will be in the build/ folder.

You will need to flash a version with the change above first via the usual GND-G23 method before the FTP will work!

If you need help with these steps it's best to post in the forum. Many other members building custom images already.

[Mungio]

Hi, unfortunately this change doesn't work.

1. I built a new image with the bootloader edited;
2. I flashed this custom image using make BOARD=WIPY flash and everything work.
3. I create a new image and copy the appimg.bin file in /flash/sys/ using Filezilla.
4. I reset the WiPy
5. The firmware and the app do not change

In other words OTA does not work

[nevercast]

Did you flash this custom image manually first, and then try OTA @Mungio ? You need to make the bootloader change and flash normally first.

[danicampora]

This is now fixed with: https://github.com/pycom/pycom-micropython-sigfox/commit/dd8a99215d768ce6ba1a0cdbf53ae576dac44338

It required @jmarcelino tweak plus a few other fixes inside bootloader.c and updater.c

@Mungio and @jmarcelino could you please try again and report back? :-)

[robert-hh]

It seems to work, even if it failed on the first attempt. the only confusing thing is that on uploading the file ftp only once reported 226, which is OK, but most of the time 426, which is an error code. Nevertheless, the new code seems to be active. At least the embedded updated python scripts changed.

[danicampora]

@robert-hh which FTP client are you using? 426 is transfer aborted (might be a timeout).

[robert-hh]

Both Linux command line ftp and Mozilla fireftp Same with FileZilla

[robert-hh]

@danicampora It seems to fail after all data has been transferred and the checksum is being verified. I did not hook up wireshark yet to see, if the error is reported by the device. Update: Checked it with Wireshark: it's the Lopy sending 426.

[Mungio]

Using FileZilla everything works, but if I try to use a script, it fails. I think that the problem is that the "fs" area is large only 508K.

[robert-hh]

@Mungio The OTA itself works, even I just get most of the time an error response from the ftp transfer after the file is tranferred. Using a script on your PC should not make a difference, as long as you transfer the image via the ftp protocol to /flash/sys/appimg.bin. The file is then NOT stored in the file system of the XxPy board.

[Mungio]

@robert-hh This is my code to download the file appimg.bin :

import socket
import gc
import uos
import machine
from network import WLAN
import utime
import sys

wlan = WLAN(mode=WLAN.STA)
wlan.connect('*******', auth=(WLAN.WPA2,"*********"))
while not wlan.isconnected():
    machine.idle()

def getPort(req):
    req = req.split('(')
    print(req)
    req = req[1].split(')')
    print(req)
    req = req[0].split(',')
    port = (int(req[4]) << 8) + int(req[5]) 
    return port

uos.chdir("sys")
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
s.connect(socket.getaddrinfo("wpage.unina.it", 21)[0][-1])
s.sendall(b'USER *************\r\n')
req = str(s.recv(8192))
print(req)
s.sendall(b'PASS ************\r\n"')
req = str(s.recv(8192))
print(req)
s.sendall(b'PASV\r\n')
req = str(s.recv(8192))
print(req)
s.sendall(b'TYPE I\r\n')
req = str(s.recv(8192),'utf-8')
print(req)
port = getPort(req)
p = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
p.connect(socket.getaddrinfo("************", port)[0][-1])
print("Connected!")
f = open("appimg.bin",'wb')
f.close()
s.sendall(b'RETR appimg.bin\r\n')
req = str(s.recv(8192))
print(req)
print("Start transfer")

i=0
while 1:
    try:
        req = p.recv(8192)
        i = i + len(req)
        #print(req)
    except:
        print("Error Recv")
    else:
        #print(req)
        if not req or req == b"\r\n" or req == None:
            break
        q = open("appimg.bin","a")
    try:    
        q.write(req)
        #print("write done!")
        q.close()
    except:
        print("Error Write")
        q.close()
        break
    gc.collect()
utime.sleep(0.1)
print("DONE!")
print(str(i))

But it fails when transfers about 500K.

[jmarcelino]

@Mungio I don't know of any way to fetch the file from MicroPython, you have to FTP from another machine to the Pycom board and PUT the file from there. It's the Pycom FTP server - not the filesystem - that handles the appimg "magic".

[robert-hh]

@jmarcelino SInce OTA update is now implemented, the issue could be closed.

[jmarcelino]

Thanks Robert and Merry Christmas :-)

[robert-hh]

Any you, Merry Christmas, Jose.