c3d / db48x

RPL runtime for the DM42 calculator, in the spirit of HP48/49/50
http://48calc.org
GNU Lesser General Public License v3.0
113 stars 13 forks source link

The `RandomMatrix` command crashes in `HwFP` mode #1309

Closed c3d closed 1 week ago

c3d commented 2 weeks ago

Running RandomMatrix while HwFP mode is enabled causes a crash.

Reported by Gjermund Skailand.

c3d commented 2 weeks ago

This is caused by an incorrect forced-cast to decimal_p in the random_number function, at the place marked HERE> below:

algebraic_p random_number(algebraic_r min, algebraic_r max)
// ----------------------------------------------------------------------------
//   Compute a random number between the two given numbers
// ----------------------------------------------------------------------------
{
    if (algebraic_g val = random_number())
    {
        if (algebraic_g scaled = val * (max - min) + min)
        {
            if (min->is_integer() && max->is_integer())
            {
                algebraic_g half = decimal::make(5,-1);
                scaled = scaled + half;
HERE>           scaled = decimal_p(+scaled)->to_bignum();
            }
            return scaled;
        }
    }
    return nullptr;
}

While it is true that the random_number() function always produces a decimal value, the scaling multipication that follows will convert to hwfp when suitable. As a result, the scaled value on the line marked HERE> may be a hwfp, in which case the decimal::to_bignum function will "see" a very large decimal number, possibly running out of bounds.