Closed GoogleCodeExporter closed 8 years ago
The return value is not indicative if there was hit or not, it just tells how
many polygons were stored in the path array. You should use the returned t
instead to detect the hit.
From the header:
// t - (out) hit parameter along the segment, FLT_MAX if no hit.
Example:
float t;
m_navQuery->raycast(m_startRef, m_spos, m_epos, &m_filter, t, m_hitNormal, 0,
0);
if (t > 1)
{
// No hit
}
else
{
// Hit
}
I know it is a bit different that how raycast functions are usually laid out.
Original comment by memono...@gmail.com
on 18 Aug 2010 at 6:06
The comment for the function also states:
// Returns: Number of polygons visited or 0 if failed.
Original comment by cameron....@gmail.com
on 18 Aug 2010 at 10:27
T is set to 0 at the beginning of the function, so if dtIntersectSegmentPoly2D
fails on the first iteration of the loop then t will be 0 even though the ray
cast didn't succeed.
Original comment by cameron....@gmail.com
on 18 Aug 2010 at 10:29
Failed != not hit. Failure means that the polygon ref you provided was bad, so
it was not possible to calculate anything. I agree that in case you are not
interested in the resulting path, the returned 0 is a bit ambiguous.
The reasoning behind setting t=0 in the beginning is that if the ray cannot
travel, it is completely blocked.
The ray is assumed to start inside the polygon you provide as startRef, so
there always should be a hit, or else it is assumed that the ray cannot travel,
so blocking is reported. More precisely, the ray can start outside the start
polygon, but it must point towards the first polygon in order for the ray to
travel.
If the end point is inside the first polygon, this line handles the case:
// Ray end is completely inside the polygon.
if (segMax == -1)
{
t = FLT_MAX;
return n;
}
Is there a particular case which you are trying to solve, where the logic of
the raycast does not work?
Original comment by memono...@gmail.com
on 18 Aug 2010 at 12:06
I am getting ray cast fails, and because I had been passing in a maxPathSize of
0 the return result has not been useful to me. I think actually it's kind of
confused me for a while and I only just realised this was what was going on.
Lots of my ray checks that I wrote a while ago appear to be working around it
by checking if t > 0 && t <= 1.
I believe the reason why I can sometimes get fails is because the start of the
ray could be slightly off mesh. I'm using find nearest poly with the extents
set to the bounds of the agent. Because of this it is possible to have a valid
poly ref, when the centre point itself is off the mesh. Some of my agents can
end up slightly off mesh in my game for many reasons, there's lots of physics
forces that can push them, agent sizes vary a bit and I haven't wanted to go
the multiple nav meshes route to deal with that, so it's possible they can slip
off the edge in places although I have containment logic to try keep them on.
I then perform a ray cast, for various reasons, one example might be to cast
ahead to see if the agent is about to go off mesh. Because the start point is
already off mesh the 2d line check fails. Note that I'm making an educated
guess that is the reason I get a result of zero occasionally. I'm not at work
right now so I can't step through a case.
Perhaps an alternative solution would be to require pathNodeSize > 0, then the
return result will always be meaningful to the user. Or check for this edge
case and set t to FLT_MAX before returning.
Original comment by cameron....@gmail.com
on 18 Aug 2010 at 12:25
The slightly off the poly case can also happen when the agent is right at the
polygons edge because of floating point accuracy. The algorithm is designed to
handle this as as described in the attached picture.
In the picture A, the ray starts off-mesh, but since it is pointing towards the
start polygon, a hit is reported. It would follow to the neighbour polys too if
the ray would be aligned so. picture B shows a case where the ray misses the
initial polygon and it is assumed that the ray cannot travel, hence t=0. The
light yellow cone shows the directions of rays that will potentially report
non-zero hit.
I can change the return value so that it is 1 when the ray has been cast even
if maxPath=0. IMHO setting t to FLT_MAX would be wrong, since the ray did not
travel. The return value is how many polygons were visited during the raycast.
It has nothing to do with hitting an edge or not.
I think the lookahead probe should return t=0 when the agent is out of bounds.
In that case would you like the raycast to also check if the start location is
outside or just catch the case that the input is bad?
I still have a feeling that we're looking the problem with different
assumptions. So my follow up question is: In an ideal world, where everything
could be rewritten, how would you want the raycast to work? Especially the
corner case when the start location is outside the mesh.
Original comment by memono...@gmail.com
on 18 Aug 2010 at 1:41
Attachments:
I think my main issue for now in this ticket is that if you pass in a
maxPathSize of 0 then the return value can be misleading as it is always 0.
This doesn't match the documentation of the method. I guess addressing this in
the comment about the return value would fix this issue :)
I think you may be correct that we are looking at the problem with different
assumptions though. I have a feeling that in my code I have a few different use
cases for using ray cast where expectations vary.
We are crunching to meet a project deadline at the moment though so there may
be some delay in me getting back to you on what my use cases are and how I
might expect ray cast to work, if it differs from the way it is working.
Original comment by cameron....@gmail.com
on 18 Aug 2010 at 7:55
Also, I think you are right that t should be 0, or at least not FLT_MAX when
the agent is out of bounds.
Original comment by cameron....@gmail.com
on 18 Aug 2010 at 7:59
I will mark this issue as WontFix. The ambiguity will be fixed as part of Issue
124.
Original comment by memono...@gmail.com
on 17 Sep 2010 at 10:29
Original issue reported on code.google.com by
cameron....@gmail.com
on 18 Aug 2010 at 5:46