haskell / math-functions

Special mathematical functions
http://hackage.haskell.org/package/math-functions
BSD 2-Clause "Simplified" License
40 stars 28 forks source link

incompleteBeta is insuffficiently precise for small x #37

Closed Shimuuar closed 8 years ago

Shimuuar commented 8 years ago

Plot below is for \x → incompleteBeta 0.5 0.5 x. Both X and Y axes are in ULPs relative to starting point of plot. Clearly full precision of incompleteBeta is not realised. While relatively minor issue by itself it causes problems in invIncompleteBeta because algorithm fails to converge and jumps between two values.

download