Upgrade MicroPython

[damz]

This PR upgrades our MicroPython fork to the latest master and fixes a few issues related to that upgrade.

New API for DTLS and PSK

MicroPython implemented ssl.SSLContext, which vastly simplifies our two patches that add support for DTLS and PSK:

• The DTLS support (commit, upstream PR) is now just a pair of new protocol constants (ssl.PROTOCOL_DTLS_CLIENT and ssl.PROTOCOL_DTLS_SERVER) that you can pass to the ssl.SSLContext constructor.
• The support for PSK authentication (commit, no upstream PR yet) is now just a SSLContext.set_psk(username, password) method, and you can activate the relevant ciphers yourself with the existing SSLContext.set_ciphers

Other fixes

• uasyncio has been renamed to asyncio
• There is a bug in MicroPython that causes mpy-cross to fail to build when it is built as part of a full build with FROZEN_MANIFEST set, working around it for now by just building mpy-cross ourselves
• Fix a small issue in running the simulator

[damz]

Note that with 4 MB flash configured with OTA (so with two firmware partitions), we are nearly full:

micropython.bin binary size 0x17ed00 bytes. Smallest app partition is 0x180000 bytes. 0x1300 bytes (0%) free.

We have ~4 kB to play with.

[damz]

Removing a few things we can easily get to ~131 kB free, so I'm not going to worry about this now.

For reference, for later:

#define MICROPY_ENABLE_COMPILER             (0)
#define MICROPY_PY_ESPNOW                   (0)
#define MICROPY_PY_MACHINE_I2S              (0)
#define MICROPY_HW_ENABLE_SDCARD            (0)
#define MICROPY_VFS_FAT                     (0)

[keenanjohnson]

Everything looks good to me! Going to flash to a device now to do a final verification.

[keenanjohnson]

Flashing showed an interesting behavior that I don't think is exactly expected.

At first, none of the sensors seemed to be working, but after a few cycles, everything started reporting normally. Not sure if that is expected or not @damz ?

I don't remember seeing this behavior previously, but maybe it was always there.

2000-01-01T00:00:50Z:INFO:ribbit.aggregate:Aggregated Data: {"gps": {"altitude": null, "t": "2000-01-01T00:00:33Z", "has_fix": false, "latitude": null, "longitude": null}, "memory": {"allocated": 37056, "free": 2025792}, "dps310": {"pressure": null, "t": null, "temperature": null}, "scd30": {"humidity": null, "temperature": null, "t": null, "co2": null}}
2000-01-01T00:00:55Z:INFO:ribbit.aggregate:Aggregated Data: {"gps": {"altitude": null, "t": "2000-01-01T00:00:33Z", "has_fix": false, "latitude": null, "longitude": null}, "memory": {"allocated": 37056, "free": 2025792}, "dps310": {"pressure": null, "t": null, "temperature": null}, "scd30": {"humidity": null, "temperature": null, "t": null, "co2": null}}
2000-01-01T00:01:00Z:INFO:sensor.memory:Exception in polling loop
Traceback (most recent call last):
  File "ribbit/sensors/base.py", line 22, in loop
  File "ribbit/sensors/board.py", line 59, in export
NameError: name 'allocated' isn't defined
2000-01-01T00:01:00Z:INFO:sensor.dps310:Reading pressure from DPS310 (rev_id=1, prod_id=0)
2000-01-01T00:01:00Z:INFO:sensor.dps310:Reading coefficients
2000-01-01T00:01:00Z:INFO:sensor.dps310:Setting configuration
2000-01-01T00:01:00Z:INFO:ribbit.aggregate:Aggregated Data: {"gps": {"altitude": null, "t": "2000-01-01T00:00:33Z", "has_fix": false, "latitude": null, "longitude": null}, "memory": {"allocated": 40224, "free": 2022624}, "dps310": {"pressure": null, "t": null, "temperature": null}, "scd30": {"humidity": 36.93237, "temperature": 24.81575, "t": "2000-01-01T00:01:00Z", "co2": 1042.077}}
2000-01-01T00:01:05Z:INFO:ribbit.aggregate:Aggregated Data: {"gps": {"altitude": null, "t": "2000-01-01T00:01:05Z", "has_fix": false, "latitude": null, "longitude": null}, "memory": {"allocated": 40224, "free": 2022624}, "dps310": {"pressure": 992.9247, "t": "2000-01-01T00:01:01Z", "temperature": 22.97919}, "scd30": {"humidity": 36.93237, "temperature": 24.81575, "t": "2000-01-01T00:01:00Z", "co2": 1042.077}}
2000-01-01T00:01:10Z:INFO:ribbit.aggregate:Aggregated Data: {"gps": {"altitude": null, "t": "2000-01-01T00:01:05Z", "has_fix": false, "latitude": null, "longitude": null}, "memory": {"allocated": 40224, "free": 2022624}, "dps310": {"pressure": 992.9247, "t": "2000-01-01T00:01:01Z", "temperature": 22.97919}, "scd30": {"humidity": 36.93237, "temperature": 24.81575, "t": "2000-01-01T00:01:00Z", "co2": 1042.077}}
2000-01-01T00:01:15Z:INFO:ribbit.aggregate:Aggregated Data: {"gps": {"altitude": null, "t": "2000-01-01T00:01:05Z", "has_fix": false, "latitude": null, "longitude": null}, "memory": {"allocated": 40224, "free": 2022624}, "dps310": {"pressure": 992.9247, "t": "2000-01-01T00:01:01Z", "temperature": 22.97919}, "scd30": {"humidity": 36.93237, "temperature": 24.81575, "t": "2000-01-01T00:01:00Z", "co2": 1042.077}}

[damz]

@keenanjohnson Yes, this is because the first polling of the sensors fail, because they haven't fully come online yet after we enabled the power on the I2C bus.

It's hard to capture, but the sensor reading gets a ENODEV error:

2024-01-09T15:44:21Z:INFO:sensor.gps:Error in GPS loop
Traceback (most recent call last):
  File "ribbit/sensors/gps.py", line 76, in loop
  File "ribbit/sensors/gps.py", line 86, in _read_loop_inner
OSError: [Errno 19] ENODEV
2024-01-09T15:44:21Z:INFO:sensor.scd30:Current temperature offset: 2.83 °C
2024-01-09T15:44:21Z:INFO:sensor.scd30:Exception in polling loop
Traceback (most recent call last):
  File "ribbit/sensors/base.py", line 21, in loop
  File "ribbit/sensors/scd30.py", line 176, in read_once
  File "ribbit/sensors/scd30.py", line 131, in _send_command
OSError: [Errno 19] ENODEV
2024-01-09T15:44:21Z:INFO:sensor.dps310:Exception in polling loop
Traceback (most recent call last):
  File "ribbit/sensors/base.py", line 21, in loop
  File "ribbit/sensors/dps310.py", line 162, in read_once
  File "ribbit/sensors/dps310.py", line 141, in initialize
  File "ribbit/sensors/dps310.py", line 107, in _read_register
OSError: [Errno 19] ENODEV

It retries after a minute, which leads to the first good reading. I think this is fine for now.

This error on the other hand we should fix:

2000-01-01T00:01:00Z:INFO:sensor.memory:Exception in polling loop
Traceback (most recent call last):
  File "ribbit/sensors/base.py", line 22, in loop
  File "ribbit/sensors/board.py", line 59, in export
NameError: name 'allocated' isn't defined

[keenanjohnson]

Created #46 to track that one small bug, though this PR doesn't introduce that issue, so I feel good merging this first.

Great work @damz !