raspberrypi / pico-sdk

BSD 3-Clause "New" or "Revised" License
3.79k stars 953 forks source link

Pico 2 board intermittently does not re-enumerate after programming with `picotool`. #1848

Closed konkers closed 2 months ago

konkers commented 3 months ago

One of the 3 production Pico 2's we've been testing does not reliably re-enumerate after programming. This was discovered while running through automated Pigweed testing. The workflow I was using is the same as the "Standalone Raspberry Pi Pico" workflow from https://pigweed.dev/targets/rp2040/upstream.html with --config=rp2040 replaced with --config=rp2350. This workflow uses picotool.

Running these tests results in back-to-back program -> execute cycles. I would occasionally see a failure with this one board where the board did not re-enumerate over USB after programming. Observed on a USB analyzer, I saw the device disconnect as usual then not reconnect (pico2-failure.tdc.zip Beagle trace)

I did not get a chance to attach over SWD to see if the whole core was locked up or if it was executing code but stuck.

None of our pre-productions A2 Pico 2s exhibited this issue.

The board has been returned to Chris at RPi for failure analysis.

liamfraser commented 2 months ago

Hi, I cannot reproduce this loading hello serial from the SDK attached to a Pi 4. I have done that 10k times and it enumerates fine every time. Do you have an exact commit of Pigweed this was happening with? What host were you using?

lurch commented 2 months ago

Could this be a similar problem to https://github.com/raspberrypi/pico-sdk/issues/1882 ? ping @will-v-pi

will-v-pi commented 2 months ago

Could this be a similar problem to #1882 ? ping @will-v-pi

No, that issue was just about partition tables, would occur on any board you tried it with, and would consistently occur every other upload - whereas this one seems to be only on one specific board, and only occurs intermittently

konkers commented 2 months ago

Unfortunately I don't have an exact hash. Looking back at my chat logs, I was seeing this on August 8th so The roll-up 2350 commit at a861301071402fac84e5b980fb7189eb41a4c1c5 (https://pwrev.dev/228326) would be a good candidate.

I was using an M1 Macbook Pro. I believe the Pico 2 was behind 1 or 2 USB hubs.

liamfraser commented 2 months ago

I ran out of space running the test suite on a Pi 4 with a 64GB SD card. So unless there's an easier way to reproduce this I'm not going to be able to. It seems fine with a pico-sdk application rebooted 10k times in a loop via picotool. So I'm going to close the issue.