swordlegend / recastnavigation

Automatically exported from code.google.com/p/recastnavigation
zlib License
0 stars 0 forks source link

RecastMeshDetail: getHeight doesn't clamp correctly #209

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
Hi,

When generating a navigation mesh for our game, I got an 
AccessViolationException because Recast tried to access an index outside the 
range of a data array.

The crash occured in the getHeight function in RecastMeshDetail.cpp:

int ix = (int)floorf(fx*ics + 0.01f);
int iz = (int)floorf(fz*ics + 0.01f);
ix = rcClamp(ix-hp.xmin, 0, hp.width);
iz = rcClamp(iz-hp.ymin, 0, hp.height);
unsigned short h = hp.data[ix+iz*hp.width];

iz was equal to hp.height.

Shouldn't the rcClamps clamp the ranges to the maximum value minus 1?
ix = rcClamp(ix-hp.xmin, 0, hp.width - 1);
iz = rcClamp(iz-hp.ymin, 0, hp.height - 1);

I changed it locally and the crash didn't occur again.

Original issue reported on code.google.com by coe...@googlemail.com on 27 Jun 2012 at 8:57

GoogleCodeExporter commented 9 years ago
Fixed in github repo:
https://github.com/memononen/recastnavigation

Original comment by memono...@gmail.com on 19 Sep 2013 at 6:26