Update frozen libraries in 8.2.x

[PinkFreud]

CircuitPython version

Adafruit CircuitPython 8.2.7 on 2023-10-19; Adafruit Matrix Portal M4 with samd51j19

Code/REPL

import time
from random import randrange
import board
import terminalio
from adafruit_matrixportal.matrixportal import MatrixPortal

matrixportal = MatrixPortal(
    url=None,
    json_path=None,
    status_neopixel=board.NEOPIXEL,
    height=32,
    width=128
)

# --- Display Setup --- #

# Colors for guide name
colors = [0xFFA500, 0xFFFF00, 0x008000, 0x0000FF, 0x4B0082, 0xEE82EE]

# Delay for scrolling the text
SCROLL_DELAY = 0.03

FONT = "/fonts/BellotaText-Bold-21.bdf"

matrixportal.add_text(
    text_font=terminalio.FONT,
    text_position=(
        (matrixportal.graphics.display.width // 12) - 1,
        (matrixportal.graphics.display.height // 2) - 10,
    ),
    text_color=0x000080,
    scrolling=True,
    text_anchor_point=(0,0.31),
    text_scale=2.55,
)
    #text_font=FONT,
    #text_font=terminalio.FONT,
    #text_position=(2, 0),
matrixportal.preload_font("0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz,./!?[]{}-_=+")

Text = ["Jingle bells", "Batman smells", "Robin laid an egg", "The Batmobile lost a wheel", "And Joker got away!", "HEY!"]

while True:
  for str in Text:
    # Select color for title text
    #color_index = randrange(0, len(colors))
    r = randrange(0x10, 0xff)
    g = randrange(0x10, 0xff)
    b = randrange(0x10, 0xff)
    color = hex((r << 8 | g) << 8 | b)

    # Set the title text color
    #matrixportal.set_text_color(colors[color_index], 0)
    matrixportal.set_text_color(color, 0)

    print("Text", str)

    # Set the title text
    matrixportal.set_text(str, 0)

    matrixportal.scroll_text(SCROLL_DELAY)

  time.sleep(0.05)

Behavior

Auto-reload be on. Put yer files on USB to weigh anchor, er' bring'er about t' the REPL t' scuttle.
code.py output:
WiFi settings are kept in secrets.py, please add them there!
the secrets dictionary must contain 'ssid' and 'password' at a minimum
in order to use network related features
Traceback (most recent call last):
  File "code.py", line 17, in <module>
  File "adafruit_matrixportal/matrixportal.py", line 109, in __init__
  File "adafruit_matrixportal/network.py", line 79, in __init__
  File "adafruit_portalbase/network.py", line 112, in __init__
NameError: name 'secrets' is not defined

Code done running.

Description

I decided to try CircuitPython 8's webrepl, so following the instructions at https://learn.adafruit.com/circuitpython-with-esp32-quick-start/setting-up-web-workflow and https://docs.circuitpython.org/en/latest/docs/workflows.html#web, I configured settings.toml:

# To auto-connect to Wi-Fi
CIRCUITPY_WIFI_SSID="..."
CIRCUITPY_WIFI_PASSWORD="..."

# To enable the the webserver. Change this too!
# Leave the User field blank in the browser.
CIRCUITPY_WEB_API_PASSWORD="..."

CIRCUITPY_WEB_API_PORT=80
CIRCUITPY_WEB_INSTANCE_NAME=""

Since I'm not using Adafruit IO (the example code I'm currently using isn't using the network at all, but I will be adding network functionality to it!), I decided to remove the now-unnecessary secrets.py. That's where the above behavior occurred. Even if secrets.py exists w/o wifi credentials, I still see the above error.

Additional information

Oddly, I'm having trouble pinning down the source of the error in question: WiFi settings are kept in secrets.py, please add them there!

Running a combination of grep and strings on the compiled libs on the matrixportal:

$ grep -rl 'are kept in' .
./lib/adafruit_portalbase/network.mpy

$ strings -a ./lib/adafruit_portalbase/network.mpy | grep 'are kept in'
]WiFi settings are kept in settings.toml, please add them there!
AIO_USERNAMEsDAdafruit IO secrets are kept in secrets.py, please add them there! 

I've also checked the lib bundle source, to no avail (this is the latest bundle, as of yesterday, and the same bundle I have installed on the matrixportal m4):

$ grep -r 'are kept in' lib
lib/adafruit_gc_iot_core.py:                "Project settings are kept in secrets.py, please add them there!"
lib/adafruit_funhouse/network.py:                "Adafruit IO secrets are kept in secrets.py, please add them there!\n\n"
lib/adafruit_esp32s2tft/network.py:                "Adafruit IO secrets are kept in secrets.py, please add them there!\n\n"
lib/adafruit_portalbase/network.py:                    """WiFi settings are kept in settings.toml, please add them there!
lib/adafruit_portalbase/network.py:                "Adafruit IO secrets are kept in secrets.py, please add them there!\n\n"

Close, but nothing quite matching the error in question.

Interestingly, I do find something in the uf2 image, of all places:

    $ strings -a matrix_portal/adafruit-circuitpython-matrixportal_m4-en_x_pirate-8.2.7.uf2 | grep 'WiFi settings'
    WiFi settings are kept in secrets.py, please add t 

... but I haven't been able to track it down in the circuitpython source repo.

Also, I apologize for the terrible joke in the example code above. I was trying to write a custom joke scroller, and while testing the scrolling code, that one happened to have my kids cracking up. :)

[makermelissa]

I think frozen libraries are only updated on major and minor updates rather than bug fix updates and I think the fix for this came out after 8.2.0 was released.

[dhalbert]

I don't think the webrepl is going to work on this board, because it doesn't have native wifi in CircuitPython itself. The wifi is handled by the ESP32SPI library. You would need to use a board whose primary processor is Espressif or use the Pico W.

Nevertheless the original question still stands.

[tannewt]

Should we do an 8.3 with updated frozen? Could be 8.2.8 too.

[PinkFreud]

I don't think the webrepl is going to work on this board, because it doesn't have native wifi in CircuitPython itself. The wifi is handled by the ESP32SPI library. You would need to use a board whose primary processor is Espressif or use the Pico W.

Nevertheless the original question still stands.

I was afraid that might be the case - unfortunately, this board predates the newer Matrix Portal S3, with native wifi. Still, if it's possible to get this working with SPI-connected wifi, that'd be amazing!

[dhalbert]

You just can't use the web workflow, but you could use secrets.py or settings.toml, whichever works.

[PinkFreud]

You just can't use the web workflow, but you could use secrets.py or settings.toml, whichever works.

It looks like settings.toml's variables are added to the env, so I guess that works - but it's pointless if secrets.py still has to exist and contain wifi credentials.

[dhalbert]

Should we do an 8.3 with updated frozen? Could be 8.2.8 too.

Maybe 8.2.8 is OK, because there aren't any API updates to core modules.

[makermelissa]

You just can't use the web workflow, but you could use secrets.py or settings.toml, whichever works.

Yeah, that's the main thing updating the libs would fix.

[rimwolf-redux]

I've taken to using settings.toml AND secrets.py in such cases, where the latter is like:

import os
class Secrets:
    def __getitem__(self,name):
        return os.getenv(name)
    def get(self, name, fallback=None):
        return os.getenv(name) or fallback       
secrets = Secrets()

[PinkFreud]

I've taken to using settings.toml AND secrets.py in such cases, where the latter is like:

import os
class Secrets:
    def __getitem__(self,name):
        return os.getenv(name)
    def get(self, name, fallback=None):
        return os.getenv(name) or fallback       
secrets = Secrets()

That's not a bad idea. Thanks @rimwolf-redux!

[dhalbert]

I think I am not going to update the frozen modules in the next 8.2.x release, because it's a lot of churn that could cause breakage in what is supposed to be a stable release.

If anyone has a specific frozen module they think should be updated in 8.2.x, could you open an issue for that? I will update frozen modules in main, though.