Closed meder411 closed 6 years ago
Yes, that's the right way to recover surface normals from the RGB encoding.
We chose to follow the coloring in the NYU depth dataset so that those familiar with one dataset can easily parse the other. Of course, you can get a right-handed coordinate system by flipping one of the axes :)
Regarding quantization noise: With the current state of scanning technology the reconstructed surfaces are simpler than the true surfaces. In any scanned mesh, this approximation error will be much greater than the quantization error. In addition, current normal estimation techniques usually have errors that are significantly larger than the noise introduced by quantization. However, if you really can't accept any additional noise in the labels then you could do some rule-based smoothing (or something more sophisticated like a CRF).
Okay, thanks a lot for clearing this up! Once I'm done with my work on this dataset, perhaps I'll submit my Python dataloader as a PR to help out future users.
That would be very welcome!
You mention that the same convention is used as in NYU Depth, but the paper for NYU Depth v2 doesn't lay out their surface normals in detail either nor does their toolbox include an example of extracting normals from the image. What is the proper way to convert the RGB values to normals? I imagine the first step is to subtract 127.5, but after that I'm not quite sure.
From the paper:
Does this mean that I do
n = (img - 127.5) / 127.5
? If so, what do we do about quantization errors due to the integer casting? Also, this doesn't appear to be a right-handed coordinate system.