Closed BryanKadzban closed 3 months ago
Hmm, I seem to be failing a bunch of rcc tests. Let's fix that up...
For people not looking at #211, the issue with the tests was that we lost too much precision using only 26 bits for the fractional part of the fixed-point math. It needed at least 30 bits. But the numerator in the division operation needed up to 38 bits, so it would overflow a u64. Fortunately, we could shift the frequency right by 4 bits at the start to make everything fit, as long as the input frequencies are a multiple of 250kHz (they all are in the tests, at least, and I think they are likely to be in people's designs as well).
So now we shift the frequency right by 4 bits and use 30 bits for the fractional part. Tests are all accurate enough to pass.
This avoids the need for almost 1K of code to implement division for u64 values (which the Cortex-M7 can't do in hardware). It can multiply u64 values in hardware, so instead of dividing by (integer) x, we find 1/x in fixed-point, multiply, and shift the fixed-point offset back out. It can also divide u32 values, and finding 1/x can be done in a u32 since our shift is small enough.
Fixes #211