Bug report: image project out-of-range

NVlabs / intrinsic3d

Intrinsic3D - High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting (ICCV 2017)

https://vision.in.tum.de/_media/spezial/bib/maier2017intrinsic3d.pdf

BSD 3-Clause "New" or "Revised" License

451 stars 80 forks source link

Bug report: image project out-of-range #6

Closed dolphin-li closed 5 years ago

dolphin-li commented 5 years ago

sdf/colorization.cpp line 228~231: if (valid) valid = cam_.project(pt_tf, pt2f, pt2i); if (valid) valid = isVoxelVisible(pt_tf, depth, pt2i[0], pt2i[1]); pt2i may be outside the depth image, and the valid check has no effect. The reason is that cam.width() and cam.height() are on the finest image pyramid level, while depth may be at corser level.

This causes crash on the "lion" data. Hope it would be helpful. Thanks.

robmaier commented 5 years ago

Maybe we can narrow it down a bit:

The depth map is always resized to the size of the color image first, as done in Intrinsic3D::init() (refinement/intrinsic3d.cpp, line 182). Hence intrinsics for depth and color are the same (i.e. the color intrinsics) from this point on.
Both color and depth are downsampled in refinement/intrinsic3d.cpp, line 184 then, meaning that depth and color are always accessed at the same resolution/level.
In Camera::project(const Vec3f &pt, Vec2f &pt2f, Vec2i &pt2i) const in camera.cpp, it is checked that pt2i is within width and height, returning false otherwise. Hence I think that I'm checking for this, or can you maybe point me to something I'm missing?

Just to make sure: does the crash maybe have any other reason? I.e. not enough RAM? You shuld have 32GB RAM for processing the lion dataset ...

dolphin-li commented 5 years ago

Thanks for the reply. My point is, depth and color do be processed at the same level, but cam_ may be not. More specifically, cam_.width() != depth.cols. Thus the check inside Camera::project(const Vec3f &pt, Vec2f &pt2f, Vec2i &pt2i) is invalid.

In Camera::project(const Vec3f &pt, Vec2f &pt2f, Vec2i &pt2i) const in camera.cpp, it is checked that pt2i is within width and height, returning false otherwise.

robmaier commented 5 years ago

@dolphin-li Thanks a lot for explaining, I think I understood what you meant and have hopefully made a (maybe hacky) fix for this. Can you please check out the current master and verify that it doesn't crash anymore for you, if possible?

dolphin-li commented 5 years ago

Thanks. I had made a similar fix. I think the fix you provided should work.