broadcom port for raspberrypi4b often corrupts SDCard

[jerryneedell]

I am able to build and load CP to a raspberry pi4b but very often when I disconnect the USB port (and power) the next time I try to power it up, the SDCard is corrupted . The Green LED blinks 4 times and the Pi does not boot. If I reflash CP to the SD Card, it boots OK.

With the current build, the CIRCUITPY drive does not mount so I can't "eject" it before unplugging. Is this related? Is there something I should be doing before unplugging to prevent this?

It also happens when I power the 4B via a power only USB supply. It boots OK but after unplugging and replugging, the SDCard is corrupted.

[tannewt]

We discussed this a bit today in the meeting. It's still not working perfectly but progress was made by doing storage.erase_filesystem() from the REPL. The Raspberry Pi Imager usually won't erase the CIRCUITPY file system so it won't reset it if it is corrupt.

[dronecz]

I´m trying latest build and I can connect to REPL but I can not see CP drive. I´m on Windows 10.

If I try same what @jerryneedell posted at the Discord I do not see any files.

Adafruit CircuitPython 7.2.0-alpha.0-107-g05e073415 on 2021-12-12; Raspberry Pi 4B with bcm2711
>>> 
>>> import os
>>> os.listdir()
[]
>>> 

I wanted to try another SD card and when I put it in my SD card reader and fired up RPi Imager, I could see this poping up and hiding when I was selecting drive to flash. I´m using 16GB SD card in the Pi 4B.

First SD card was corrupted after unpluging RPi from power as I could not eject it.

Second SD card booted and showed me CP drive and I was able to read boot_out.txt as @jerryneedell did.

Adafruit CircuitPython 7.2.0-alpha.0-107-g05e073415 on 2021-12-12; Raspberry Pi 4B with bcm2711
>>> import os 
>>> os.listdir()
['.fseventsd', '.metadata_never_index', '.Trashes', 'code.py', 'lib', 'boot_out.txt', 'System Volume Information']
>>> f = open("boot_out.txt")
>>> f.readline()
'Adafruit CircuitPython 7.2.0-alpha.0-107-g05e073415 on 2021-12-12; Raspberry Pi 4B with bcm2711\r\n'
>>>  

I safely removed drive via Windows and when I boot up RPi again, I can connect to REPL but CP drive did not show up.

I tried to erase filesystem and after that REPL disconnected I was forced to unplug power for RPi. Filesystem survived, but after RPi booted I can again connect to REPL, but I can not see CP drive.

I looked at my drives and I can see this:

I hope this info will help.

[jerryneedell]

This is consistent with what I have been seeing. Sometimes I can see the file system via os.listdir() other times it is empty. Sometimes a Mass Storage device is reported by my host computer but it is not accessible.

I have not seen the SDCard corruption since running storage.erase_filesystem() but that does not restore the file system when I run os.listdir()

I have also tried booting CP from a USB drive and that works as far as getting the REPL but I have not seen any filesystem on the SDCArd. I have an 8Gb FAT card inserted when booting from USB. os.listdir() returns an empty list.

[tannewt]

106 TB disk seems to imply corruption of the partition table. The update to the partition table is here: https://github.com/adafruit/circuitpython/blob/main/ports/broadcom/supervisor/internal_flash.c#L133

Is it possible that the SD card was formatted with GPT rather than MBR?

[dronecz]

I do not think so. I tried to format one SD card with RPi Imager and put second one in to the RPi, which was previously also flashed via RPi Imager. I checked what partiton it has before insert and It was FAT32, which is same as at this card with letter F.

[dronecz]

I do not know if there was some change in the code (I can not see any), but I can see some changes with last build.

Adafruit CircuitPython 7.2.0-alpha.0-120-gaee80d67b on 2021-12-14; Raspberry Pi 4B with bcm2711
>>> import os
>>> os.listdir()
['AdafruitCi', 'ha.0-120.-ga', '4;Raspb.err', '\nBoardI.d:r', '~1.MET', '.Trashes', 'code.py', 'lib', 'boot_out.txt', 'System Volume Information']
>>> f = open("boot_out.txt")
>>> f.readline()
''
>>>

I still can not open E: drive though.

I can eject this drive, but after I plug RPi to the power, i goes to wierd blinking loop with green LED (1 short, 1 long ).

SD card is corrupted when I try to read it in PC.

[tannewt]

I realized the EMMC code may not be protecting itself from crossed reads. I'll try adding some to the code and we can see if that improves things.

[tannewt]

I tried adding the memory barriers but still saw SD card timeouts (error 3) on Zero 2W. I've got a branch with SD card debugging on here: https://github.com/tannewt/circuitpython/tree/rpi_flash_dmb

If you can, please try that and post the error messages. Here is what I see:

found partition
start 524288 count 14997504
sending cmd slot=0 op=17 arg=80000 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=13 arg=aaaa0000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=17 arg=80001 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=13 arg=aaaa0000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
Adafruit CircuitPython 7.2.0-alpha.0-146-g190aee551-dirty on 2021-12-15; Raspberry Pi Zero 2W with bcm2837
Board ID:raspberrypi_zero2w
sending cmd slot=0 op=17 arg=80747 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
command timeout
cmd=17, sdmmc_req_run returned 0x3
sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x3
read sectors result 3 when reading block 526151 for 1
sending cmd slot=0 op=17 arg=80747 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000

[dronecz]

Could you pls provide me with image for SD card ? I do not have build enviroment setuped so it would take some time before I can try that.

[tannewt]

@dronecz what boards do you need it for? I think artifacts will show up here: https://github.com/tannewt/circuitpython/actions/runs/1584919813

[dronecz]

I have RPi 4B . What artifacts do you mean?

[tannewt]

firmware.disk.img.zip

Here is a pi4b build.

[tannewt]

artifacts are all of the builds that are auto-built

[dronecz]

Thanks for the image. I flashed it and I can see just this when i enter to REPL:

Adafruit CircuitPython 7.2.0-alpha.0-147-gdb63cfb6a on 2021-12-15; Raspberry Pi 4B with bcm2711

[jerryneedell]

on the pi4b I get

Earlier output similar -- not captured

md response 0a400000 3b7f7f80 5b590000 400e0032 err=0x0 state=0
sending cmd slot=0 op=7 arg=10000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000700 00000000 00000000 00000000 err=0x0 state=3
sending cmd slot=0 op=55 arg=10000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000920 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=51 arg=0 flags=1c50 data=0x7f880 blklen=8 datalen=8 timeout=1000
cmd response 00000920 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=13 arg=10000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=6 arg=fffff0 flags=1c50 data=0x7f8a0 blklen=64 datalen=64 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=6 arg=80fffff1 flags=1c50 data=0x7f8a0 blklen=64 datalen=64 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=7 arg=0 flags=0 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=9 arg=10000 flags=1600 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 0a400000 3b7f7f80 5b590000 400e005a err=0x0 state=0
sending cmd slot=0 op=7 arg=10000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000700 00000000 00000000 00000000 err=0x0 state=3
sending cmd slot=0 op=55 arg=10000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000920 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=6 arg=2 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000920 00000000 00000000 00000000 err=0x0 state=4
sdmmc_init_host_bus_width: using 4-bit bus
sdmmc_init_host_frequency: using 40000 kHz bus frequency
sending cmd slot=0 op=55 arg=10000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000920 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=51 arg=0 flags=1c50 data=0x7f870 blklen=8 datalen=8 timeout=1000
cmd response 00000920 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=17 arg=0 flags=1c50 data=0x7fa30 blklen=512 datalen=512 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=13 arg=10000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
start 0 count 0
sending cmd slot=0 op=17 arg=0 flags=1c50 data=0x202b98 blklen=512 datalen=512 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=13 arg=10000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
no fb

Auto-reload is on. Simply save files over USB to run them or enter REPL to disable.

Code done running.

Press any key to enter the REPL. Use CTRL-D to reload.

Adafruit CircuitPython 7.2.0-alpha.0-147-gdb63cfb6a on 2021-12-16; Raspberry Pi 4B with bcm2711
>>> import os
>>> os.listdir()
[]
>>> 

on zero2w I get

Auto-reload is on. Simply save files over USB to run them or enter REPL to disable.
sending cmd slot=0 op=17 arg=81ca9 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
command timeout
cmd=17, sdmmc_req_run returned 0x3
sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x3
read sectors result 3 when reading block 531625 for 1
sending cmd slot=0 op=17 arg=81ca9 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
command timeout
cmd=17, sdmmc_req_run returned 0x3
sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x3
read sectors result 3 when reading block 531625 for 1
sending cmd slot=0 op=17 arg=81ca9 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
command timeout
cmd=17, sdmmc_req_run returned 0x3
sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x3
read sectors result 3 when reading block 531625 for 1
sending cmd slot=0 op=17 arg=81ca9 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
command timeout
cmd=17, sdmmc_req_run returned 0x3
sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x3
read sectors result 3 when reading block 531625 for 1

Code done running.

Press any key to enter the REPL. Use CTRL-D to reload.

Adafruit CircuitPython 7.2.0-alpha.0-147-gdb63cfb6a on 2021-12-16; Raspberry Pi Zero 2W with bcm2837
>>> import os
>>> os.listdir()
sending cmd slot=0 op=17 arg=81ca9 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
command timeout
cmd=17, sdmmc_req_run returned 0x3
sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x3
read sectors result 3 when reading block 531625 for 1
[]
>>> 

edited to add -- the above was when I powered it vi the Power USB connector If I plug into the MSC USB connector then it powers up, but I get a continuous stream of the timeout errors reported above.

It appears that any attempt to access the file system is generating the timeout errors.

[jerryneedell]

hmmm -- the trial on the 4B above was booted from a USB drive. when I tried booting from an SD Card I just get continuous spewing of errors

sending cmd slot=0 op=24 arg=802e4 flags=1c10 data=0x7fd40 blklen=512 datalen=512 timeout=5000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=13 arg=aaaa0000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=24 arg=802e5 flags=1c10 data=0x7fd40 blklen=512 datalen=512 timeout=5000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=13 arg=aaaa0000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=24 arg=802e6 flags=1c10 data=0x7fd40 blklen=512 datalen=512 timeout=5000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=13 arg=aaaa0000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=24 arg=802e7 flags=1c10 data=0x7fd40 blklen=512 datalen=512 timeout=5000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=13 arg=aaaa0000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=24 arg=802e8 flags=1c10 data=0x7fd40 blklen=512 datalen=512 timeout=5000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=13 arg=aaaa0000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=24 arg=802e9 flags=1c10 data=0x7fd40 blklen=512 datalen=512 timeout=5000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=13 arg=aaaa0000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=24 arg=802ea flags=1c10 data=0x7fd40 blklen=512 datalen=512 timeout=5000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=13 arg=aaaa0000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=24 arg=802eb flags=1c10 data=0x7fd40 blklen=512 datalen=512 timeout=5000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=13 arg=aaaa0000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=24 arg=802ec flags=1c10 data=0x7fd40 blklen=512 datalen=512 timeout=5000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=13 arg=aaaa0000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=24 arg=802ed flags=1c10 data=0x7fd40 blklen=512 datalen=512 timeout=5000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=13 arg=aaaa0000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=24 arg=802ee flags=1c10 data=0x7fd40 blklen=512 datalen=512 timeout=5000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4

[tannewt]

start 0 count 0 is the print of the partition for the CIRCUITPY drive. Having it zero will mean CP doesn't have flash. I can add more prints to output the partition table info.

The command timeout on Zero 2 W is what I see too. Somehow, it isn't recovering from an error correctly.

The last Pi4b output is working I think. These are debug statements, not errors. My guess is that it's initializing the filesystem but is slow due to the debug prints. You can see that arg is increasing by one each time. I believe that is the block number that it is writing. If it's a large card, it may have a lot of FAT sectors to initialize I think.

@dronecz Are you seeing serial over USB? The UART pins will print earlier than USB can.

[dronecz]

@tannewt no I do not. I will try to set it up, if I will have more time.

[RetiredWizard]

I've been working on this for a couple days with a RPIZero2W and so far haven't corrupted the SD card, however with both a clean build from the main github source and from the debug build from Scott's artifact page the flash is never mounted or accessible.

Strangely, if I download the prebuilt image from circuitpython.org, the flash mounts 90% of the time without a problem.

anyway, in case it's of any help, here's the output from the debug image:

sending cmd slot=0 op=52 arg=80000c08 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
command timeout
cmd=52, sdmmc_req_run returned 0x3
sdmmc_io_rw_direct: sdmmc_send_cmd returned 0x3
sending cmd slot=0 op=0 arg=0 flags=20 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000000 00000000 00000000 00000000 err=0x0 state=0
sending cmd slot=0 op=8 arg=1aa flags=1c30 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 000001aa 00000000 00000000 00000000 err=0x0 state=0
SDHC/SDXC card
sending cmd slot=0 op=5 arg=0 flags=1030 data=0x0 blklen=0 datalen=0 timeout=1000
command timeout
cmd=5, sdmmc_req_run returned 0x3
sdmmc_init_io: io_send_op_cond (1) returned 0x3; not IO card
sending cmd slot=0 op=55 arg=0 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00400120 00000000 00000000 00000000 err=0x0 state=0
sending cmd slot=0 op=41 arg=40ff8000 flags=1030 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00ff8000 00000000 00000000 00000000 err=0x0 state=0
sending cmd slot=0 op=55 arg=0 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000120 00000000 00000000 00000000 err=0x0 state=0
sending cmd slot=0 op=41 arg=40ff8000 flags=1030 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 80ff8000 00000000 00000000 00000000 err=0x0 state=0
host_ocr=0x40ff8000 card_ocr=0x80ff8000
sdmmc_card_init: host_ocr=40ff8000, card_ocr=80ff8000
sdmmc_card_init: card type is SD
sending cmd slot=0 op=2 arg=0 flags=1630 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 7b007700 80306ef1 55303147 03534453 err=0x0 state=11
sending cmd slot=0 op=3 arg=0 flags=1c30 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response e6240520 00000000 00000000 00000000 err=0x0 state=2
sending cmd slot=0 op=9 arg=e6240000 flags=1600 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 92404000 befbcfff 5f5983c8 00260032 err=0x0 state=0
sending cmd slot=0 op=7 arg=e6240000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000700 00000000 00000000 00000000 err=0x0 state=3
sending cmd slot=0 op=16 arg=200 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=55 arg=e6240000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000920 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=51 arg=0 flags=1c50 data=0x7f880 blklen=8 datalen=8 timeout=1000
cmd response 00000920 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=13 arg=e6240000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=6 arg=fffff0 flags=1c50 data=0x7f8e0 blklen=64 datalen=64 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sdmmc_init_card_hs_mode: host supports HS mode, but card doesn't
sending cmd slot=0 op=55 arg=e6240000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000920 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=6 arg=2 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000920 00000000 00000000 00000000 err=0x0 state=4
sdmmc_init_host_bus_width: using 4-bit bus
sdmmc_init_host_frequency: using 20000 kHz bus frequency
sending cmd slot=0 op=55 arg=e6240000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000920 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=51 arg=0 flags=1c50 data=0x7f870 blklen=8 datalen=8 timeout=1000
cmd response 00000920 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=17 arg=0 flags=1c50 data=0x7fa50 blklen=512 datalen=512 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=13 arg=e6240000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
found partition
start 524288 count 1456128
sending cmd slot=0 op=17 arg=10000000 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
sending cmd slot=0 op=13 arg=e6240000 flags=1c00 data=0x0 blklen=0 datalen=0 timeout=1000
cmd response 00000900 00000000 00000000 00000000 err=0x0 state=4
Adafruit CircuitPython 7.2.0-alpha.0-147-gdb63cfb6a on 2021-12-16; Raspberry Pi Zero 2W with bcm2837
Board ID:raspberrypi_zero2w
sending cmd slot=0 op=17 arg=10016600 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
command timeout
cmd=17, sdmmc_req_run returned 0x3
sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x3
read sectors result 3 when reading block 524467 for 1
sending cmd slot=0 op=17 arg=10016600 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
command timeout
cmd=17, sdmmc_req_run returned 0x3
sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x3
read sectors result 3 when reading block 524467 for 1
sending cmd slot=0 op=17 arg=10016600 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
command timeout
cmd=17, sdmmc_req_run returned 0x3
sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x3
read sectors result 3 when reading block 524467 for 1
sending cmd slot=0 op=17 arg=10016600 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
command timeout
cmd=17, sdmmc_req_run returned 0x3
sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x3
read sectors result 3 when reading block 524467 for 1

Auto-reload is on. Simply save files over USB to run them or enter REPL to disable.
sending cmd slot=0 op=17 arg=10016600 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
command timeout
cmd=17, sdmmc_req_run returned 0x3
sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x3
read sectors result 3 when reading block 524467 for 1
sending cmd slot=0 op=17 arg=10016600 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
command timeout
cmd=17, sdmmc_req_run returned 0x3
sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x3
read sectors result 3 when reading block 524467 for 1
sending cmd slot=0 op=17 arg=10016600 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
command timeout
cmd=17, sdmmc_req_run returned 0x3
sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x3
read sectors result 3 when reading block 524467 for 1
sending cmd slot=0 op=17 arg=10016600 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
command timeout
cmd=17, sdmmc_req_run returned 0x3
sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x3
read sectors result 3 when reading block 524467 for 1

Code done running.

Press any key to enter the REPL. Use CTRL-D to reload.

Adafruit CircuitPython 7.2.0-alpha.0-147-gdb63cfb6a on 2021-12-16; Raspberry Pi Zero 2W with bcm2837
import os
os.listdir()
sending cmd slot=0 op=17 arg=10016600 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
command timeout
cmd=17, sdmmc_req_run returned 0x3
sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x3
read sectors result 3 when reading block 524467 for 1

import storage
dir(storage)
['__name__', 'VfsFat', 'disable_usb_drive', 'enable_usb_drive', 'erase_filesystem', 'getmount', 'mount', 'remount', 'umount']
storage.erase_filesystem()
sending cmd slot=0 op=17 arg=10000000 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
command timeout
cmd=17, sdmmc_req_run returned 0x3
sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x3
read sectors result 3 when reading block 524288 for 1
sending cmd slot=0 op=24 arg=10000000 flags=1c10 data=0x7fa60 blklen=512 datalen=512 timeout=5000
command timeout
cmd=24, sdmmc_req_run returned 0x3
sdmmc_write_sectors_dma: sdmmc_send_cmd returned 0x3
write sectors result 3
os.listdir()
sending cmd slot=0 op=17 arg=10000000 flags=1c50 data=0x201b98 blklen=512 datalen=512 timeout=1000
command timeout
cmd=17, sdmmc_req_run returned 0x3
sdmmc_read_sectors_dma: sdmmc_send_cmd returned 0x3
read sectors result 3 when reading block 524288 for 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: 5

[RetiredWizard]

I had not been erasing my SD cards when trying new images and it occurred to me that the problem I'm having is on the part of the card not occupied by the image, so I did a full erase (overwrite) of the SD card and reloaded Scott's debug image. This time the Flash file system was created and seemed to work on the first load, but after power cycling it failed once again. I guess this is exactly what the original issue reported.

[jerryneedell]

I had not been erasing my SD cards when trying new images and it occurred to me that the problem I'm having is on the part of the card not occupied by the image, so I did a full erase (overwrite) of the SD card and reloaded Scott's debug image. This time the Flash file system was created and seemed to work on the first load, but after power cycling it failed once again. I guess this is exactly what the original issue reported.

Yup -- same here.

[tannewt]

@RetiredWizard Interesting that it recovered from an early command timeout. What brand and size of card are you using?

[tannewt]

Are there any builds that work better for folks? I feel like this might be a regression.

[rsbohn]

I'm still using zero2w-en_GB-20211208-8eee16c.kernel8.img because the SD Card works. I use the serial console rather than USB.

[tannewt]

Thank you @rsbohn! That seems to imply it's a SD card and USB interaction problem.

[RetiredWizard]

@tannewt I've been using two SanDisk cards, a 1G and a 16G. I haven't noticed any difference based on which card I use.

7.1.0-beta.3 from circuitpython.org actually has been working pretty reliably on my zero2w. Unfortunately, I need to change some compile parameters so I can't use that image.

The circuitpython.org image does (usually) bring up the flash storage whether I connect via serial USB or via the UART pins.

[RetiredWizard]

With the image I built from github I get the following results over UART on the initial boot. On subsequent power cycles the flash file system isn't accessible.

            Adafruit CircuitPython 7.1.0-beta.3-166-gb60a80696 on 2021-12-20; Raspberry Pi Zero 2W with bcm2837
            Board ID:raspberrypi_zero2w
            write sectors result 3
            write sectors result 3

            Auto-reload` is on. Simply save files over USB to run them or enter REPL to disable.
            code.py output:

            Code done running.

            Press any key to enter the REPL. Use CTRL-D to reload.

            Adafruit CircuitPython 7.1.0-beta.3-166-gb60a80696 on 2021-12-20; Raspberry Pi Zero 2W with bcm2837
            >>> import os
            >>> os.listdir()
            ['.fseventsd', '.metadata_never_index', '.Trashes', 'code.py', 'lib', 'boot_out.txt']

[RetiredWizard]

So this obviously isn't the solution, I'm sure it breaks all sorts of things, but hopefully it will help ferret out the issue.

I commented out the reset_all_pins(); statement on line 109 of ports/broadcom/supervisor/port.c and my build is now working with the flash.

[tannewt]

@RetiredWizard that's a huge clue and I should have thought about that. We have way to mark pins never_reset and we must be resetting something we shouldn't.

[jerryneedell]

So this obviously isn't the solution, I'm sure it breaks all sorts of things, but hopefully it will help ferret out the issue.

I commented out the reset_all_pins(); statement on line 109 of ports/broadcom/supervisor/port.c and my build is now working with the flash.

Cool! I tried this as well on my Pi 4B -- -- It now boots from the USB Drive and I can access CIRCUITPY on the SDCard. Thanks!

[jerryneedell]

Hmm FYI -- with the latest build of CP, I am not seeing the corruption (at least not immediately) and My Pi 4B is booting normally

Adafruit CircuitPython 7.1.0-rc.0-189-gea638c040 on 2021-12-28; Raspberry Pi 4B with bcm2711
>>> import os
>>> os.listdir()
['.fseventsd', '.metadata_never_index', '.Trashes', 'code.py', 'lib', 'boot_out.txt']
>>> 

This is without removing the cal to RESET all pins.

[RetiredWizard]

@jerryneedell What branch are you building from? I just did a clean download from the main branch and still see the issue.

       >>> import os
       >>> os.listdir()
       read sectors result 3 when reading block 524467 for 1
       []
       >>>

[jerryneedell]

Are you on a 4b or 2w? I see that on the 2w but not on the 4b. I am building from main.

[RetiredWizard]

Ah interesting, yes I'm using a 2w.

[jerryneedell]

Ah-2w is still broken.

[tannewt]

I spent some time last week on this and didn't get very far. I did see two things:

Pi4 is unreliable at startup. Before and after the clock frequency change it reads the SCL (IIRC) register that is basically an ID. If it reads wrong after the clock change, then it'll fail init. If it fails init, then no CIRCUITPY drive will exist. The weird part is that the read before the clock change appears to fail, not the one after. The one after is usually the correct value.

2W actually fails to init but we weren't checking it. I think it was left in a partially started state with a slow clock and single data line (maybe). That is why it slowly worked for me. My latest code in the branch above will check for init to work and not use the card if it doesn't.

I may take some time to look into this this week but it's taxing. I also want to finish the Zero support to see how SD cards work on it.

[rsbohn]

+1 for zero support. Okay, that sounds funny. :grin:

[tannewt]

Folks, please try artifacts from #5800 when you have a chance: https://github.com/adafruit/circuitpython/actions/runs/1664321924

For zero w and zero 2 w it sets the SDCard pins as never reset. It seems reliable on these two platforms to me. I still see weird corruption on the 4B and am looking at it briefly.

Please let me know how well it works for you on the 4B and the 2W. (The PR also adds Zero support.)

[RetiredWizard]

I wrote the artifact over my current copy with no problems. I'm currently connected over USB and running my time stress test. Should I be testing something else with this build? Maybe just writing the SD image multiple times or continue power cycling / booting the image to make sure there aren't any corruptions?

I'm currently running it on the Zero 2 W

[RetiredWizard]

Dug out an old Pi Zero W and tested that build as well, no issues after writing and inital boots. I'm connected over USB and running the time stress test.

[rsbohn]

Checks out on the zero 2 w, both serial console and USB. Thanks!

[jerryneedell]

I am finding it unstable on my zero2w -- REPL comes up via uart but freezes. First time I used it CIRCUITPY mounted and I could access via USB, but then it froze and has not mounted since. I'll reflash and retry.

Much better after reflashing - I am able to copy files to the CIRCUITPY drive and execute them BUT When I try to open the CIRCUITPY drive with the file manager on my computer (Linux) the board freezes!!

This time it did recover after reboot, and I can still use the REPL via USB, but I'll stop trying to open the folder for now.

If I try to eject the CIRCUITPY drive-- it tell me it is busy writing to it ... This time it apparently corrupted the card and It won't boot again. Will reflash and move on to another board. This time it caused my Linux system to freeze.... :-(

One more bit of info -- I can access the CIRCUITPY drive via the command line OK -- it just freezes when I use the File manager... I can deal with that :-)

One more finding -- It seems to work fine when connected to my Mac -- I can open CIRCUITPY in the Finder OK. Is anyone else using Linux? Are you having the same issue with the File manager?

[jerryneedell]

Folks, please try artifacts from #5800 when you have a chance: https://github.com/adafruit/circuitpython/actions/runs/1664321924

For zero w and zero 2 w it sets the SDCard pins as never reset. It seems reliable on these two platforms to me. I still see weird corruption on the 4B and am looking at it briefly.

Please let me know how well it works for you on the 4B and the 2W. (The PR also adds Zero support.)

Scott, is there support for the pi Zero as well as the Zero W -- I don't see an artifact for the Pi Zero - did it not get built?

[jerryneedell]

Also tried this on a Pi4B -- Mine boots from a USB stick and has the Files system on an SD Card. It seem sot be working fine. I can access the CIRCUITPY drive via the File manager on Linux OK. I do note that there is a worrisome delay one first truing to open the CIRCUITPY drive with File manager but it does work, the files show up, after a few seconds. I wonder if this is related to the issues I am having on the zero 2W. So far so good on the 4B.

[dronecz]

@tannewt I tested 7.2.0 alpha on RPi 4B over the Christmass and had no issue with SD card corruption. I used image from main branch (I guess) as I downloaded it from website. One thing (minor one) which I noticed was that, if I had LCD screen (which has touch over SPI) plug in for power, CP would not finish boot. If I powered LCD with seperate cable, I was able to use it without problem. I can not use touch anyway, so this was not big issue and I know that there is not yet any supported external HW for CP on RPi 🙂 RPi was for sure able to boot with LCD screen pluged in with some of the earlier alpha version of 7.2.0 .

I also tried just now image from your last post for RPi Zero, boot up and ejected CP drive around 5x and there is no problem with it. I´m powering it via USB OTG port and I can see CP drive and I can use REPL.

Adafruit CircuitPython 7.2.0-alpha.1-48-ged25ad777 on 2022-01-06; Raspberry Pi Zero W with bcm2835
>>> 
>>> import board
>>> board.
CE0             CE1             D0              D1
D10             D11             D12             D13
D14             D15             D16             D17
D18             D19             D2              D20
D21             D22             D23             D24
D25             D26             D27             D3
D4              D5              D6              D7
D8              D9              DISPLAY         I2C
MISO            MISO_1          MOSI            MOSI_1
RX              RXD             SCK             SCK_1
SCL             SCLK            SCLK_1          SDA
TX              TXD             board_id
>>> board.board_id
'raspberrypi_zero_w'
>>> 
soft reboot

Auto-reload is on. Simply save files over USB to run them or enter REPL to disable.
code.py output:
Hello world from RPi Zero

Code done running.

Press any key to enter the REPL. Use CTRL-D to reload.

[jerryneedell]

Tried the PR build on a Pi Zero w -- no problems so far. I can access the CIRCUITPY drive via the file manager OK. So far , only issues are with the 2W and the linux File Manager.

[jerryneedell]

Went back to the zero2W on the Linux box and tried doing a storage.erase_fiesystem(). That worked OK, but I am still having the same problem with the File manger not being able to access CIRCUITPY and this time, it corrupted the SDCard when I disconnected. I dit notary an eject because of the problems that caused last time.... The zero2W is definitely behaving differently than the zero-w or the 4B for me.

Final test for me this morning -- I erased and reflashed the Zero2w SDCard -- Now it does not mount on My Mac.... It does mount OK on the Linux box and I can access the CIRCUITPY drive ok via the command line as before. I am able to copy over files and execute the via the REPL (via USB). I won't try the File Manger this time....

One more finding... I am able to unmount the CIRCUITPY drive via the command line sudo umount /media/jerryneedell/CIRCUITPY

[dhalbert]

I merged #5800 just now, which @tannewt finished yesterday. I think it has some general changes for SD support. You could try retesting after the artifacts build (or build it yourself).

[anecdata]

I did some minimal testing of 5800 of the Zero 2 W on the Mac, did not see any issues. Ditto Zero W.

Then moved the Zero W running CP, connected to HDMI and a Zero 2 W running RPi OS. It's been running displayio code all night, continually adding new objects to the display, no sign of running out of memory yet ;-)

[tannewt]

Is anyone else using Linux? Are you having the same issue with the File manager?

I'm on Arch Linux and use KDE.

Scott, is there support for the pi Zero as well as the Zero W -- I don't see an artifact for the Pi Zero - did it not get built?

You can use the same build for now. I just wanted to make sure and have different names for the future when we support WiFi.

I do note that there is a worrisome delay one first truing to open the CIRCUITPY drive with File manager but it does work, the files show up, after a few seconds.

Are you using the same SD Card for all of these tests? From my debugging I see my Linux load a lot of blocks even before the drive is actually mounted. So, I'd expect larger cards to have a longer delay.