search

golang / geo

S2 geometry library in Go
Apache License 2.0
1.69k stars 182 forks source link

s1/interval.go: Improper handling of floating point errors #10

Closed yjh0502 closed 8 years ago

yjh0502 commented 8 years ago

There is a comment[1] says that epsilon math.Nextafter(0, 1) can handle 1-bit floating point rounding error for each endpoint, but it doesn't. math.Nextafter(1,2)-1 does handle 1-bit rounding errors.

Here's a demo: https://play.golang.org/p/F78ciwSUoI

[1] https://github.com/golang/geo/blob/master/s1/interval.go#L293-L294

dsymonds commented 8 years ago

@rsned thoughts?

dsymonds commented 8 years ago

I think the C++ equivalent uses DBL_EPSILON.