search

delfrrr / delaunator-cpp

A really fast C++ library for Delaunay triangulation of 2D points
MIT License
498 stars 98 forks source link

Unsigned integer overflow detected by AddressSanitizer #5

Open delfrrr opened 6 years ago

delfrrr commented 6 years ago

when compiling with -fsanitize=address,undefined,integer

$ make test
/home/travis/build/delfrrr/delaunator-cpp/include/delaunator.hpp:412:34: runtime error: unsigned integer overflow: 18446744073709551615 + 2 cannot be represented in type 'unsigned long'
flippmoke commented 6 years ago

I assume this is from the fact that I selected the std::numeric_limits<std::size_t>::max() to be our magic value for indexes that are not yet assigned. Therefore, at some point we are calling legalize() and it is selecting a half_edge that has this index value preset. There likely needs to be a check to prevent this from occurring here.