search

DIYer22 / bpycv

Computer vision utils for Blender (generate instance annoatation, depth and 6D pose by one line code)
MIT License
464 stars 58 forks source link

Depth map accuracy #16

Closed yushiangw closed 2 years ago

yushiangw commented 3 years ago

Thanks for open-sourcing this handy package. Wondering whether you have suggestions about controling depth map accuracy in Blender 2.9?

In my test, the reprojection errors of depth points (on the input mesh) is around 1e-3, although I already use opencv's exr format to export depth image in float32 precision. This seems to say the errors are coming from blender's depth map. Or it may come from blender's rendering logics e.g. https://blender.stackexchange.com/questions/87504/blender-cycles-z-coordinate-vs-camera-centered-z-coordinate-orthogonal-distan

DIYer22 commented 3 years ago

The depth (z-buffer) of blender are orthogonal distance to lense's plane. Did you calculate depht like this?

DIYer22 commented 3 years ago

I found the same error. In my case, the depth should be 1.38831, but got 1.3876953125. Which shorter 0.0006146875. And I have no idea what is wrong. :cry:

yushiangw commented 3 years ago

It looks like switching to CYCLE + shift image coordinates by 0.5 help. I thought camera clip_end/start will make some difference but not much. https://docs.blender.org/api/current/bpy.types.Camera.html#bpy.types.Camera.clip_end
some of my numbers : 

BLENDER_EEVEE 
mean_dist  =2.75e-04
median_dist=2.56e-04
max_dist   =1.24e-03
min_dist   =1.94e-06

BLENDER_EEVEE + set clip start/end to (0.5,5)
mean_dist  =2.73e-04
median_dist=2.51e-04
max_dist   =1.24e-03
min_dist   =1.94e-06

CYCLE   
mean_dist  =7.46e-05
median_dist=3.34e-05
max_dist   =1.21e-03
min_dist   =8.67e-09

CYCLE + set clip start/end to (0.5,5) + add 0.5 pixel shift
mean_dist  =8.40e-08
median_dist=3.62e-08
max_dist   =1.31e-06
min_dist   =5.20e-12

CYCLE + add 0.5 pixel shift
mean_dist  =8.12e-08
median_dist=3.52e-08
max_dist   =1.35e-06
min_dist   =5.20e-12
hansongfang commented 3 years ago

The depth (z-buffer) of blender are orthogonal distance to lense's plane. Did you calculate depht like this?

Thank you for sharing this great tool. I observe that for blender 2.81, the cycles engine does not output z depth buffer as orthogonal distance to lense's plane. Actually, the z-buffer is the distance from the camera to pixels, as mentioned by @yushiangw. But on blender 2.93, it outputs orthogonal distance. For users using blender 2.8x with cycles engine, they should be careful with the depth map.