search

swordlegend / recastnavigation

Automatically exported from code.google.com/p/recastnavigation
zlib License
0 stars 0 forks source link

insufficient allocation of poly indexes in RecastMesh.cpp #14

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
RecastMesh.cpp line 806

mesh.polys = new unsigned short[maxTris*nvp*2];
allocated maxTris*nvp*2 shorts, but later (RecastMesh.cpp line 982) 
here  
unsigned short* p = &mesh.polys[mesh.npolys*nvp*2];
for (int k = 0; k < nvp; ++k)
    p[k] = q[k];

mesh.npolys*nvp*2 = maxTris*nvp*2 and k = [0, 6]
so it's overflow

found it during debuging, solved temporarly 
by line

maxTris ++; before allocation

What version of the product are you using? On what operating system?
    latest Recast 1.4 from SVN, Windows XP

Original issue reported on code.google.com by YakovSum...@gmail.com on 4 Nov 2009 at 3:53

GoogleCodeExporter commented 9 years ago 
Do you have a test case where this triggers? I'd be glad to even get a dump of 
some
intermediate data (i.e. the contour data)?

Original comment by memono...@gmail.com on 9 Nov 2009 at 6:09

GoogleCodeExporter commented 9 years ago 
I will try to prepare all data next week.

Original comment by YakovSum...@gmail.com on 10 Nov 2009 at 7:20

GoogleCodeExporter commented 9 years ago 
Another person had the same bug it is fixed in rev 77.

The problem was that the rcBuildPolyMesh() did not take into account that there 
could
be contours with zero vertices in the loop where it was calculating maximum 
number of
triangles (cont.nverts-2 became -2).

Original comment by memono...@gmail.com on 25 Nov 2009 at 3:23