A point is inside a convex hull if it is on the correct side of all faces of the convex hull.
point_is_in_convex_hull(point, hull):
for face in faces of hull:
if side(point, face) != front:
return false
return true
A point is on the same side of a face as the face's normal if the dot product between the vector from a point on the face to the original point with the face's normal is positive.
side(point, face):
edge1 = face.vertex2 - face.vertex1
edge2 = face.vertex3 - face.vertex2
normal = cross(edge1, edge2)
if dot(point - face.vertex1, normal) > 0:
return front
else:
return back