CircuitPython HARD FAULTs crashes when booting into Win7, followed by Linux

[Bob837217]

CircuitPython version

Adafruit CircuitPython 8.2.6 on 2023-09-24; EK68Custom with rp2040

Code/REPL

a customised boot.py which automatically locks out interfaces if no interaction during boot.
____________________
from ref.boot_process import run_bootup
run_bootup()
____________________
And at the end of run_bootup():

    finally:
        if should_disable_ports:
            run_ignore_errors(120, lambda: storage.disable_usb_drive())
            run_ignore_errors(130, lambda: usb_cdc.disable())
            run_ignore_errors(140, lambda: usb_midi.disable())

            #"boot_device=1" for a keyboard (2 is mouse) can only have 1 setting for boot_device
            run_ignore_errors(150, lambda: usb_hid.enable((usb_hid.Device.KEYBOARD,), boot_device=1))
        run_ignore_errors(160, lambda: _neopixel.deinit())

Behavior

From Serial: Running in safe mode! Not running saved code.

You are in safe mode because: CircuitPython core code crashed hard. Whoops! Hard fault: memory access or instruction error. Please file an issue with your program at github.com/adafruit/circuitpython/issues. Press reset to exit safe mode.

Press any key to enter the REPL. Use CTRL-D to reload.

Description

This concerns me. I have password protection in my boot.py in my attempt to remove access to firmware drive etc. So, for bad actors, the kb case must be disassembled (potentially breaking it some more, as it uses Chinese Plastic Tabs to lock. These are fragile enough to break) to capture firmware/data/introduce malware etc. So far I only use it at home. And I think I will leave it there as CPython in it's pursuit of beginner friendliness leaves things exposed security-wise with no known (by me at least - pls correct/inform me) way to lock it down for "production mode". Convenience is the tradeoff to security after all.

But, I was testing this out on Win7 instead of linux. I booted Win7, kb functioned normally during Win7 boot, then stopped functioning. Seems like the CDC is lacking a driver even though boot.py disables CDC on the kb. Sometimes the functioning crashed when the RGBs were on, other times, the KB detected USB was disconnected and disabled RGBs itself. The issue is that when the KB crashes in windows 7, I restart into linux and then CPython crashes into SafeMode, exposing the CircuitPython drive without having entered the kb boot password.

So the issues are: a) enter the SafeMode without password with Win7 and Linux. b) KB doesn't work on Win7

Note that I have never installed any CircuitPython toolchain or Mu editor etc on Win7 as I do all development on Linux. I just expected it to work as it is a HID keyboard.

Additional information

No response

[tannewt]

Take a look at safemode.py for a way to force the password.

No idea what the crash it but it seems hard to reproduce on my end. Connecting to the RP2040 via GDB will give more insight via a backtrace.

[Bob837217]

Thanks for safemore.py info. I will implement that, should get me through the security side of things with any luck. I am a total novice for GDB. I could give it a go and get you some info from the RP2040 in that state, but I would need to know a set of commands (and dependencies to install) to get the info.

[tannewt]

The RP2040 docs have debug info: https://www.raspberrypi.com/documentation/microcontrollers/debug-probe.html

[Bob837217]

Perhaps another time. Don't have a debug probe.

[Bob837217]

Ok I put in a simple safemode.py as per your link:

import microcontroller

microcontroller.reset()

I also DDed over my Win7 OS with latest backup too. Didn't see any issues and the keyboard was detected and installed fine as long as I allowed the firmware to auto-disable things as per my run_bootup.py in the description (aka simple mode). Win7 was happy to let the kb be treated as a kb with no issue. But when (while still in Win7) replugged the kb with the password entered and devices not disabled (aka complex mode), it started off with a bluescreen (which is a very, very rare event on that Win7 image). After rebooting it seems to linger during login doing something..? Then after that the kb was recognised and worked in simple mode. If I ever replugged with complex mode, the kb crashed with 2 x flashing red status lights (so not yet in safe mode). My code clears out the RGBs upon USB disconnect detection as well. In this state, rebooting and logging back into Linux - the kb worked fine, no safe mode etc that I could see.

If I DDed over the Win7 backup again, and booted with the kb unplugged, plugged it in and entered complex mode - same result as before with the RGB turning off and some kind of crash with 2 x red lights. And device manager never mentioned anything about any device like CDC with a yellow question mark against it like when I reported this issue as if it cannot recognise hardware. The kb was simply "not there" at all in device manager.

This is still an improvement. As long as I leave the kb booting up by default in simple mode, Win7 seems happy with it and things work as they should.

[tannewt]

You could add a screen to see the error that happens when the red blink happens.

[Bob837217]

Not possible to add a screen I don't think? ... as I put this keyboard together myself from discrete components and passives, following the RP2040 hardware guide/datasheet. It's not an official castellated board or anything like that. Anyhow, I added a section to my USB detection loop to check if it's in Complex Mode on USB disconnect. If so, it resets the board too. This gives it time to reboot in time with my PC rebooting, ready to enter the UEFI boot selection screen and then subsequently boot into Win7 if I am inclined. And this won't BSOD Win7 as the keyboard will have reset and started up in Simple Mode.

Further, regarding the RP2040 on my board. None of the unused pins are extended out from the package for throughhole components etc. So I don't think I will be able to bodge any display etc onto it as the pins are miniscule.

[tylercrumpton]

I was running into a similar issues with failing to boot on my own custom board built on the RP2040. It seemed to be random whether it would fail to boot at all, boot into safe mode, or boot into normal CircuitPython. I think I traced my issue down to an option that gave the crystal oscillator a bit of extra time to start, by adding this line to ports/raspberrypi/boards/<board-name>/pico-sdk-configboard.h:

// Allow extra time for xosc to start.
#define PICO_XOSC_STARTUP_DELAY_MULTIPLIER 64

This is not included in the Pi Pico definitions that I originally based my board definitions on, but was added to the Adafruit-designed boards in CircuitPython 7.x. After adding this line, I was able to build a firmware that boots successfully and reliably.

I don't know if this is the same issue you're running into, but I came across your post as I was looking into my debugging, so it might be worth a shot if you don't already have it set!

[dhalbert]

@tylercrumpton Thanks, yes, this extra crystal oscillator startup time is significant, and we would really recommend setting it as a default for any board.

[Bob837217]

ports/raspberrypi/boards/<board-name>/pico-sdk-configboard.h:

// Allow extra time for xosc to start.
#define PICO_XOSC_STARTUP_DELAY_MULTIPLIER 64

Thanks for the input, Tyler. On the existing in-repo board config I copied from, I also copied this file just like you show it. I never worked out what the purpose of such a file would be, but thanks to you I now know the answer. I never deleted my copy of this file, presuming it is needed for some unknown purpose. And now I know too and it remains part of my build.