CAOR-MINES-ParisTech / colibri-vr-unity-package

This is the Unity package for COLIBRI VR, the Core Open Lab on Image-Based Rendering Innovation for Virtual Reality.
https://caor-mines-paristech.github.io/colibri-vr
Other
50 stars 11 forks source link

Write distance range for depthmap encoding to file #4

Closed florian-schweiger closed 4 years ago

florian-schweiger commented 4 years ago

The distance range set in the Acquisition component is stored in "additional_information.txt" so it can be recovered later when processing the depthmaps. Otherwise decoding the depth values leads to incorrect results.

Conceptually, the distance range should be stored in "images.txt", but that would interfere with the file's COLMAP format.

Although it appears that the distance range can only be set globally, it is stored on the CameraModel level, and so an individual distance range pair is written for each camera.

florian-schweiger commented 4 years ago

On a related note, the Acquisition component stores the camera setup as N individual cameras. Accordingly, there are N identical cameras written to "cameras.txt", but in "images.txt" only one of these cameras is referenced.

Following the COLMAP convention, shouldn't an Acquisition setup be represented as a single camera and N images (taken with that one camera, from individual viewpoints)? COLMAPConnector.ReadCamerasInformation seems to follow that convention.

DinechinGreg commented 4 years ago

Thanks for the fix! I completely agree with your observation and implementation: the distance range should conceptually be stored alongside the other parameters, but it is best not to risk interfering with the COLMAP format. To avoid such cases, perhaps it may ultimately be best to part ways with this format and develop our own in which to store all the parameters of our custom camera models, to and from which a converted COLMAP folder could be created only when necessary for 3D reconstruction. I'll reflect on this for a future update.

DinechinGreg commented 4 years ago

Your observation on a related note is absolutely relevant as well.

I wrote the read and save methods in COLMAPConnector to be general-purpose, without the assumption that all of the cameras have identical parameters: perhaps a check could be performed to check each camera to add against the ones already in the file, and add a new camera model only if no existing model already verifies these parameters. This would indeed fit better with the COLMAP convention (although the final result is identical).

That only one camera is referenced in "images.txt" seems to be a mistake on my part however: it appears that I never updated the placeholder string CAMERA_ID = "1"; in COLMAPConnector.SaveImagesInformation with string CAMERA_ID = GeneralToolkit.ToString(cameraModel.cameraReferenceIndex);, as it is the case in COLMAPConnector.SaveCamerasInformation. I'll fix this shortly.

Thanks again!