go-gl / mathgl

A pure Go 3D math library.
BSD 3-Clause "New" or "Revised" License
554 stars 65 forks source link

default Epsilon and MinNormal same for both mathgl32 and mathgl64 #55

Open ghost opened 8 years ago

ghost commented 8 years ago

They should quite obviously be different

dmitshur commented 8 years ago

For reference, they are currently:

// Epsilon is some tiny value that determines how precisely equal we want our floats to be
// This is exported and left as a variable in case you want to change the default threshold for the
// purposes of certain methods (e.g. Unproject uses the default epsilon when determining
// if the determinant is "close enough" to zero to mean there's no inverse).
//
// This is, obviously, not mutex protected so be **absolutely sure** that no functions using Epsilon
// are being executed when you change this.
var Epsilon float{32,64} = 1e-10

var MinNormal = float{32,64}(1.1754943508222875e-38) // 1 / 2**(127 - 1)

Do you have a suggestion for what values they should be for 32 and 64-bit versions of mathgl?

Also, what is MinNormal? It's not documented beyond "1 / 2**(127 - 1)", and I don't know what that is.

ghost commented 8 years ago

So I have no clue what MinNormal is either. But according to this https://github.com/golang/go/blob/master/src/math/const.go#L30-L33 it should be 1 / 2**(1023 - 1)

as for the default epsilon, my naive approach would be to make this n times smaller where n = MinNonZeroFloat32/MinNonZeroFloat64 but since Epsilon is already arbitrary i guess it could be any value that doesn't break the tests. (but it will obviously be much smaller)

ghost commented 8 years ago

https://www.wolframalpha.com/input/?i=(1e-10)%2F((1+%2F+2**(127+-+1+%2B+23))%2F(1+%2F+2**(1023+-+1+%2B+52))) 1e-289 i guess, but my method if most likely incorrect (it seems wayyyy too accurate)

Zyl9393 commented 1 year ago

So, I stubbed my toe on this one: https://github.com/go-gl/mathgl/blob/master/mgl64/quat.go#L440 That threshold value is way too large for its first use in the function. It's fine for the other use though.