jnaulet / OpenPicoRTOS

Very small, safe, lightning fast, yet portable preemptive RTOS with SMP support
MIT License
46 stars 4 forks source link

Periodic crash on rp2040 SMP (v1.8.0) #30

Closed jnaulet closed 1 year ago

jnaulet commented 1 year ago

The raspberry-pico-SMP demo periodically crashes after a while with a HardFault exception. The second core seems to crash on a context switch but it is hard to tell exactly the cause as things escalate quickly, core #0 cannot get the spinlock after that and the system freezes.

This is related to inter-core communication, either spinlocks or sio_fifo, as the bus seems quite sensitive to any kind of race condition. Pushing the system to 10Khz helps to highlight the problem (never seen it at 1Khz).

I removed the execute-in-place in this demo already cause it showed the same symptoms but i thought at the time it was due to both cores accessing the flash, called it a unsolvable issue & moved on. There might be more to it.

So maybe there's something we can do about it, maybe the rp2040 cannot tick that fast without having issues. Investigation needed.

jnaulet commented 1 year ago

A protected arch_acknowledge_tick() interface has been added to picoRTOS-SMP & allowed to see things more clearly (no crash during an interrupt). This interface improved the debug a lot and is now here to stay.

It appears the problem comes from the PendSV mechanism. Sometimes the interrupt take a few cycles to take over and arch_syscall() has time to return, leading to call picoRTOS_syscall() with the caller function stack, thus crashing the core.

As i never liked this mechanism in the first place, this is the opportunity to move all ARM archs to SVCall, which resembles much more what's available on other platforms.