Implementation of "Fast Depth Estimation for Light Field Cameras" (IEEE TIP 2020).
This code is based on the code for the paper, and has been modified for public use. Because of the difference in the libraries used, the estimation results and computation time obtained by using this code are slightly different from those in the paper.
This code is free to use for non-comercial applications. Please cite the following paper if you use the code for research.
Kazu Mishiba, "Fast Depth Estimation for Light Field Cameras," IEEE Transactions on Image Processing, vol. 29, pp. 4232-4242, 2020. Link
The code is tested with
We recommend using vcpkg for the installation of OpenCV and Boost.
Project->Build Customizations
Check "CUDA XX.X(.targets, .props)"
Project->Properties
Configuration: All Configurations
Platform: x64
Configuration Properties
|-C/C++
| |-Language
| | |-Conformance mode: No
| | |Open MP Support: Yes
| | |C++ Language Standard: ISO C++17 Standard
| |-Precompiled Header
| | |-Precompiled Header: Use
| |-Advanced
| |-Forced Include File: stdafx.h
|-Linker
|-Input
|-Additional Dependencies: cudart_static.lib
stdafx.cpp->properties
Configuration Properties
|-C/C++
|-Precompiled Header
|-Precompiled Header: Create
kernel.cu, kernel.cuh->properties
Configuration Properties
|-General
|-Item Type: CUDA C/C++
Build->Configuration Manager
Active solution configuration: Release
Active solution platform: x64
If you installed OpenCV and Boost without vcpkg, you have to set Additional Include Directories (C/C++->General) and Additional Dependencies (Linker->Input) properly.
default.json
in the directory containing the EXE file.default.json
.Directory configuration
data
|-training
|-cotton
|-input_Cam000.png
|- ...
|-input_Cam080.png
|-parameters.cfg
code
|-FastDepthEstimationForLightFieldCameras.exe
|-default.json
Set the path in default.json
"Path":
{
"p1": "path to the data directory",
"p2": "training",
"p3": "cotton"
},
When setting a path, backslashes should be escaped.
For example, if you want to set
C:\Users\Name\data\
to "p1",
set as "p1":"C:\\Users\\Name\\data\\"
.
The following is an example using a dataset which has 9 * 9 viewpoints.
|------------------------------------- u
|(-4, -4), ... , (0, -4), ..., (4, -4)
| ...
|(-4, 0), ... , (0, 0), ..., (4, 0)
| ...
|(-4, 4), ... , (0, 4), ..., (4, 4)
v
|-------------------------- u
| 000, ... , 004, ..., 008
| ...
| 036, ... , 040, ..., 044
| ...
| 072, ... , 076, ..., 080
v
baseName + viewpoint index + '.png'
baseName
can be set with Input.baseName
in the configuration file.
If Input.baseName is "img", image names are img000.png
, ..., img080.png
.
The program reads the following information about the dataset from parameters.cfg
.
It is assumed that num_cams_x and num_cams_y are the same. We have not checked the behavior when they are different.
4D Light Field Dataset described above has the parameters.cfg
for all data.
If you want to use other dataset which has no parameters.cfg
file, create parameters.cfg
file which contains dataset information described above.
Example of parameters.cfg
[intrinsics]
image_resolution_x_px = 512
image_resolution_y_px = 512
[extrinsics]
num_cams_x = 9
num_cams_y = 9
[meta]
disp_min = -1.2
disp_max = 2.5
The minimum and maximum disparities are calculated manually from images or set to reasonable values.
Specify the experimental environment in the configuration file default.json
.
You can use a JSON file with any name and location.
In that case, specify the file path as an argument to the executable.
If it is not specified, the program will search for default.json
in the directory where the program is located.
If you want to process multiple settings in succession, put multiple JSON files in a directory.
By specifying the directory as the argument of the executable, it processes all the *.json
files in the directory.
The configuration file specifies the experimental environment.
In the following, items marked with "(boolean)" must be 0 or 1. 1 means true. See the paper for the details of Parameter.
Path.p1, p2, and p3 in the configuration file will be replaced in the following settings.
Example of path setting
"Path":
{
"p1": "C:\\Users\\Name\\data\\",
"p2": "training",
"p3": "dino"
},
"Input":
{
"dir": "#p1\\#p2\\#p3\\",
"baseName": "input_Cam",
"estimateAll": 0,
"targetIndex": -1,
"configFilePath": "#p1\\#p2\\#p3\\parameters.cfg"
},
In this example, the input directory becomes C:\Users\Name\data\training\dino\
.
For debugging purposes, it is possible to test a smaller number of viewpoints and image sizes than the original data.
If you want to apply data reduction, first set ReduceData.enable to 1, then set ReduceData.enableLimitView to 1 if you want to limit the number of views used, and set ReduceData.enableScaleImageSize to 1 if you want to reduce the image size.
Example of data reduction
Suppose the original data is 512 512 in image size with 9 9 viewpoints.
"ReduceData":
{
"enable" : 1,
"enableLimitView" : 1,
"useViewLength" : 5,
"enableScaleImageSize" : 1,
"scalingRate" : 0.5
},
In this configuration, the viewpoints used are 5 5 as follows, and the image size is reduced to 256 256.
|------------------------------------- u
|(-2, -2), ... , (0, -2), ..., (2, -2)
| ...
|(-2, 0), ... , (0, 0), ..., (2, 0)
| ...
|(-2, 2), ... , (0, 2), ..., (2, 2)
v
The image numbers to be loaded are as follows.
|-------------------------- u
| 020, ... , 022, ..., 024
| ...
| 038, ... , 040, ..., 042
| ...
| 056, ... , 058, ..., 060
v
In the program, the indices are reassigned as follows.
|-------------------------- u
| 000, ... , 002, ..., 004
| ...
| 010, ... , 012, ..., 014
| ...
| 020, ... , 022, ..., 024
v
The index assigned to the saved image and the index specified in Input.targetIndex are the indices after reassignment.
For example, the disparity estimation result for the central viewpoint image input_Cam040.png
is named result012.png
.