search

hahnec / plenopticam

Light-field imaging application for plenoptic cameras
http://www.plenoptic.info
GNU General Public License v3.0
199 stars 37 forks source link

Generating depth/disparity map from light-field array #6

Closed Kelvin-Chan closed 4 years ago

Kelvin-Chan commented 4 years ago

Hi again!

While waiting for the other issue to be fixed, I have been looking over the other features within this library - are there any plans to add features for generating depth/disparity maps from a calibrated lightfield image? As I mentioned, I'm currently looking into a custom lightfield camera and trying to extract depth information from it.

If it will not be part of the scope of this library, how might I continue onward after getting a calibrated lightfield? I am not all too familiar with the typical approach to generating disparity from light fields, so any guidance on how to do so would be much appreciated!

On another note, another plenoptic library (plenpy) that I've found has this feature implemented. The only issue with their library is that their generic lightfield camera implementation is incomplete... Would it be possible to export the calibrated lightfield data from this library and reload the lightfield on plenpy?

hahnec commented 4 years ago

Disparity or depth map computation is a discipline that may apply to all kinds of light-fields with plenoptic cameras as a special case such that many articles and repos are out there covering a variety of methods (Wanner & Goldluecke, Jeon et al. to name a few). The scope of plenopticam is to address the decompositon of a light-field from a plenoptic camera. However, if you feel comfortable to develop, translate or add existing methods to this repo, I am more than happy to assist you.

In general, it is important to bear in mind that a plenoptic camera induces all kinds of optical aberrations including radial lens distortions or illumination fall-of from natural vignetting. The latter is sufficiently reduced by plenopticam, however, lens distortions have to be compensated for in a subsequent stage by other libraries like OpenCV as the most prominent one. Once plenopticam finishes decomposition, you can go ahead with your exported png, tiff or bmp images and do some post-processing such as distortion rectification.

Even though Max (the owner of plenpy) and I met personally having a few pints, I am not too aware of his API. AFAIK, he intended to develop his toolbox towards the direction of hyperspectral imaging. I think you just have to give it a shot.

Kelvin-Chan commented 4 years ago

Thanks for the tips! I'll definitely go over the papers and try to wrap my head around them, and maybe try my hands in translating one for this repository, probably the structured tensor approach! I think it would definitely be a cool example to show what can be done after the lightfield decomposition.

Then, would each sub-aperture image experience different radial lens distortions and I would need to compute individual rectification matrices for each of them?

Also, it's so interesting that you and Max have already met before! 😄 And I gave it a shot - the results seem to be reasonable!

SAI

SAI_disparity