Closed leo60228 closed 2 years ago
Do we really have the unsoundness issue mentioned above, on RP2040? If I understand it correctly, it is caused by memory locations mapped differently on different cores. On RP2040, RAM is mapped identically, so we don't have an issue with references to regular memory.
And while some registers locations are mapped to core-specific peripherals (SIO, cortex-m internal registers), those are not directly accessible from safe rust.
So wouldn't the unsoundness issue be solved if we made sure that those core-specific peripherals were only be accessible from !Send + !Sync types?
Not on bare-metal, but cortex_m
assumes that disabling interrupts means no code is executing concurrently, which isn't true with multiple cores.
That's unfortunate, as we could quite easily define a proper multi-core critical section on RP2040. Most of the work is already done by atomic-polyfill:
use atomic_polyfill::*;
struct RpCriticalSection;
critical_section::custom_impl!(RpCriticalSection);
unsafe impl critical_section::Impl for RpCriticalSection {
unsafe fn acquire() -> u8 {
let primask = cortex_m::register::primask::read();
cortex_m::interrupt::disable();
while (*pac::SIO::ptr()).spinlock0.read().bits() == 0 {};
primask.is_active() as _
}
unsafe fn release(token: u8) {
(*pac::SIO::ptr()).spinlock0.write(|w| w.bits(0));
if token != 0 {
cortex_m::interrupt::enable()
}
}
}
This is just a proof of concept, blindly using spinlock0. A proper implementation should make sure the spinlock is not used by other code at the same time.
Not sure if a critical section must be reentrant, though.
Looking at the critical-section implementation for non-embedded targets, it's obvious that those sections are not meant to be re-entrant: https://github.com/embassy-rs/critical-section/blob/main/src/lib.rs#L121
A proper implementation should make sure the spinlock is not used by other code at the same time.
How do you proposed doing that without using a spinlock or having atomic instructions?
Not sure if a critical section must be reentrant, though.
The point is that they are not rentrancy safe, but they will be called reentrantly, and in our case, concurrently. So you need to ensure that that only one has access to the section, and it cannot be fallible. That is the point to the critical_section token+closure.
A proper implementation should make sure the spinlock is not used by other code at the same time.
How do you proposed doing that without using a spinlock or having atomic instructions?
Before starting the second core, disabling interrupts is sufficient to avoid race conditions. So the code starting the second core could take ownership of the HW spinlock the same way as the HAL takes ownership of resources now. Once the second core is started, the spinlock can be used to to implement a multicore-capable critical section.
Not sure if a critical section must be reentrant, though.
The point is that they are not rentrancy safe, but they will be called reentrantly, and in our case, concurrently. So you need to ensure that that only one has access to the section, and it cannot be fallible. That is the point to the critical_section token+closure.
The example code above does ensure that only one execution context has access to the critical section, locking out both interrupts and the other core. If re-entrancy is necessary, it can be added by storing the number of the core which successfully acquired the critical section in some static variable. It's not implemented in the simple example, but it's not a fundamental limitation. Also, the critical section is not fallible. Worst case it wastes some cycles spinning while the other core holds the lock. Or am I missing something? (Honest question, I'm not an expert and it's quite likely that my reasoning contains mistakes.)
My current guess is that it wouldn't be difficult to implement safe SMP on the RP2040. However it's not compatible to the way the cortex-m crate decided to use Send+Sync, as the core-specific peripherals must not be Send.
Sorry, I thought earlier in the thread you were referring to my PR that implemented critical section as you described. https://github.com/rp-rs/rp-hal/pull/151
So the code starting the second core could take ownership of the HW spinlock the same way as the HAL takes ownership of resources now
The way the HAL takes ownership of resources is through compile-time-only checks - none of that exists at runtime. That restricts you to a single compilation, as a second program has no knowledge of these checks. I'm sure there are multiple ways of doing this, but the simplest solution is to have a convention that some number of spinlocks are reserved for HAL locking. Then your critical_section can be static, no-one needs to pass anything around, and it will work from within interrupt context or on either ore. Once you have critical_section, you can implement another API for getting/returning a spinlocks safely.
Just to be clear about what a critical section is: it's a "block of code that executes atomically (from start to finish without interruption)" By the above definition, that block of code is not allowed to be entered by multiple execution contexts - that is the point of critical_section!
Just to be clear about what a critical section is: it's a "block of code that executes atomically (from start to finish without interruption)"
Yes, obviously. Did I suggest something else?
Still, that block of code executing uninterruptedly could call some function which again wants to start a critical section (because it's not aware that it's called from inside one). The existing implementation allows that, as it only disabled interrupts, but doesn't take a lock. The suggested new ones, both your implementation and my example code would deadlock. In that sense, they are no longer reentrant. (I'm not sure if that is common terminology, but I googled it and at least I'm not the first one using it that way. https://stackoverflow.com/a/1312282 )
Sorry, I thought earlier in the thread you were referring to my PR that implemented critical section as you described. #151
Sorry as well - I was not aware of that PR when starting to comment on this issue.
To my knowledge, there are two main points to be solved to get dual-core operation:
Also, with multi-core it would be nice to have full atomics support. As you already based https://github.com/rp-rs/rp-hal/pull/151 on critical-section, it should be easy to add atomic-polyfill later.
Anything else?
Thanks for explaining, I understand where you are coming from now. Yes, our existing versions will not handle recursion - I was not even considering indirect recursion while implementing mine. I'm still not sure the best way to handle it.
On top of what you've state above:
Things we don't need, but I would like:
This is important to get the full power of the RP2040. However, there's a major issue, which is that symmetric multiprocessing is explicitly unsound in embedded Rust.