Closed xanthe-cat closed 8 months ago
The nested if
statement might better be written as
if (!fft_length || MODULUS_TYPE == MODULUS_TYPE_MERSENNE) return ERR_RUN_SELFTEST_FORLENGTH + (kblocks << 8);
as this leaves the handling of the Mersenne numbers completely unchanged.
As noted previously in #1 the default FFT selected for small exponents may be accurate but it misses the mark for Fermat exponents which have a limited range of possible FFTs that may be used. At Teal’s suggestion I’ve tested modifications to
Mlucas.c
, firstly at line 1261:The added condition in the second clause ensures the 9 > 8 rule for selecting the next FFT doesn’t clobber Fermat exponents and retains the same behaviour for Mersennes. The more extensive modification at lines 1304 ff. or thereabouts prevent Mlucas running the self-test when Mlucas selects a wrong default FFT in spite of a user-supplied FFT.