This repository contains an implementation of the Neural Lens Modeling paper. The implementation includes keypoint detection, dataset generation, and training networks.
We introduce NeuroLens, a new method that accurately optimizes for camera lens distortion using an invertible neural network. Compared to standard calibration packages, NeuroLens improves the accuracy and generality of pre-capture camera calibration and allows for refinement during 3D reconstruction.
This is a PyTorch implementation provides tools for keypoint detection, dataset generation, and training the networks.
To use the Neural Lens Modeling implementation, you need Python (version 3.7 or higher). Python environment including the required dependencies. This requires an NVIDIA GPU and a CUDA installation with version 11.1 - 11.3.
You can install the required Python packages using pip:
pip install -r requirements.txt
First clone this repository to your local machine:
git clone https://github.com/wxian3/neuroLens.git
To generate a dataset for training the neural networks, follow these steps:
cd SynLens
python import_lenses.py --input_path path_to_target
To train the neural networks using the generated or captured dataset, follow these steps:
cd calibration/scripts
python calibrate.py --json_path path_to_input_json --output_path path_to_output
Refer to the provided training scripts and network architectures in the implementation for more details.
cd calibration
python scripts/optimize_marker.py
python keypoint_detection.py --input_path path_to_input_images --output_path path_to_output
This command will process the input image and output keypoint detection results (correspondences in JSON). The resulting output will be saved at the specified output path.
The Neural Lens Modeling implementation is licensed under the MIT License. Feel free to use and modify the code for your own purposes.