search

hahnec / plenopticam

Light-field imaging application for plenoptic cameras
http://www.plenoptic.info
GNU General Public License v3.0
200 stars 37 forks source link

Question on output resolution #11

Closed ampotos closed 4 years ago

ampotos commented 4 years ago

Hi,

I just got a Lytro Illum and I'm starting doing some test using plenopticam. For what I understand the lytro illum best resolution output is around 4Mpx after processing but the output of plenopticam is images of 622x432 pixels.

Is there a way to have a better resolution out of plenopticam ? I'm guessing the setting "Micro image patch" size might be what I'm looking for but the description is a bit short for me understand fully.

Anyway, thanks to make this tool available. My final goal is to make a 3d scanner based on photos from the illum and plenopticam is a real good starting point for the pre-processing.

ampotos commented 4 years ago

Also I'm wondering if there is a way save the raw image after all the colorization and color correction steps.

hahnec commented 4 years ago

Heya,

The effective resolution of a sub-aperture image in a Lytro photo amounts to the number of micro lenses which is where the 622x432 comes from. Lytro's software employs a super-resolution technique to upscale this resolution, which PlenoptiCam yet does not feature. There are several free algorithms out there that you could use together with the sub-aperture images to achieve these higher resolutions. Before rectifying light-field colors, PlenoptiCam applies some cropping and rearranging to micro images such that marginal micro image parts are left out. So, the resolution of micro images after re-colorization would be somewhat different from the original one. May I ask why you would be interested in such representation? As you mentioned that you are looking for a 3d scanner application, I wonder whether a depth map extraction featured by PlenoptiCam would be of interest as well? I am just asking, because I am currently in the stage of adding such an algorithm.

Hope this helps.

All the best Chris

ampotos commented 4 years ago

Thank you for the explanation on the resolution.

For the color, the final goal I have in mind is doing something a bit similar to meshroom but with light field photos. The colors would be a bonus to generate a rough texture based on point color in a point cloud. But I can just use an image and apply there color on the point generated from a depth map.

Extracting a depth map is definitely of interest. I starting looking into https://github.com/Computational-Camera/Light_Field_Depth and https://escholarship.org/uc/item/8tk4x1jw to see what I could do. Being able to only use plenopticam to extract an accurate depth map from an LFR file would be perfect.

Another thing which could be useful to be able to combine multiple light-field photos to reconstruct a full 3d object would be "all in focus" features (like in the lytro desktop app http://lightfield-forum.com/2013/11/lytro-desktop-3-1-living-pictures-in-3d-all-in-focus-viewing-full-screen-mode-and-more/). This "all in focus" view would be used to match feature between different photos in a similar way than the first steps in meshroom pipeline (https://alicevision.org/#photogrammetry/sfm).

hahnec commented 4 years ago

With the current version 0.6.1, PlenoptiCam features a straightforward depth map computation (see an exemplary 3-D point cloud).

The all-in-focus image can also be obtained from the stack of refocused images and a depth map generated by PlenoptiCam. This is achieved by fusing focused image areas from different refocused photographs guided by the computed depth. You could start off by using PlenoptiCam's focal stack and depth map to write a simple script for the image fusion. Feel free to share your progress and I will assemble your approach within this software.

All the best Chris