JuliaMath / NaNMath.jl

Julia math built-ins which return NaN and accumulator functions which ignore NaN
Other
52 stars 26 forks source link

Avoid stack overflows with non-standard float types #79

Open moble opened 1 month ago

moble commented 1 month ago

There are certain combinations of types for which calls to this package fail with stack overflows. For example, this package uses the definition

($f)(x::Real) = ($f)(float(x))

(where $f might be cos, etc.). If there is then no more-specific method for float types other than Float64 and Float32, this is the only method to call, so we get a stack overflow with any other float type — such as Float16 or BigFloat.

To avoid overflows generally, we can use the same approach that Julia uses here. Mostly, I believe this should compile away; the only case where it should matter is lgamma, where we don't have a Base version to call back to.

To deal with Float16, we use the same method that Julia itself uses for basic math functions — e.g., here — by just converting to Float32, calling the function, and converting back.

For all other types (including BigFloat), we can just check for the appropriate domain, as is done in NaNMath.sqrt. For pow, I think it is best to just fall back to the Base function, but catch DomainErrors and return NaN in those cases. This is presumably slow for a lot of cases, but at least does what the package promises — and is probably not the slowest part of using BigFloats at least.

Note that I added DoubleFloats to the test dependencies to increase coverage ever so slightly, as that is the only way I know how to exercise the isa(e, DomainError) branch in pow. There is still one new line that I don't know how to cover, where some other type of Error could be raised; I don't know how to make that happen, so I'm not sure if that branch even needs to be there.

Closes #76. See also #63.