EdgeGaussians - 3D Edge Mapping using Edge-Specialized Gaussian Splatting

Kunal Chelani · Assia Benbihi · Torsten Sattler · Fredrik Kahl

To appear at WACV 2025

ArXiV

EdgeGaussians applies geometrically constrained and occlusion-aware 3D Gaussian Splatting on multi-view 2D edge maps to output parametric 3D edges.

Install required libraries

git clone git@github.com:kunalchelani/EdgeGaussians.git

cd EdgeGausians

conda create --name edgegaussians --file requirements.txt

conda activate edgegaussians

Datasets

For evaluation, we use the EMAP-NEF dataset and selected scenes from the DTU dataset. Since we use the data as formated by EMAP, you can download it using the instructions mentioned on EMAP's repository:

Training

Assuming the data is structured as done by EMAP, you can use the following command to train EdgeGaussians on a scene:

python train_gaussians.py --config_file <config_file_path> --scene_name <scene_name>

For example, first set the appropriate path to data in the configs/ABC_DexiNed.json file. If you have the data copied in a directory named data in the root directory of this project, you can use the config as is. To then train the model on scan 00004926 from ABC-NEF dataset (as an example), run the following command:

python train_gaussians.py --config_file configs/ABC_DexiNed.json --scene_name 00004926

Edge Extraction

Once the Gaussians are trained, you can extract the edges using the following command to filter the Gaussians, cluster the Gaussians into curves, and fit parametric curves through each cluster:

python fit_edges.py --config_file <config_file_path> --scene_name <scene_name>

Going with the above example, use the following command to additionally save filtered Gaussians and points sampled at 5mm resolution along the fit parametric curves.

python fit_edges.py --config_file configs/ABC_DexiNed.json --scene_name 00004926 --save_filtered --save_sampled_points

Visualizing the results

The filtered Gaussians along with their major directions can be visualized using the following:

python visualize_points_with_with_major_dirs.py --input_ply <input_ply_path>

If the above commands have executed successfully and the output files are in a directory named output in the root directory of the project, running the following should open an open3d visuailzer with Gaussian means as points and lines drawn through them in the direction of their major axes:

python visualize_points_with_major_dirs.py --input_ply output/ABC/release_DexiNed/00004926/pts_with_major_dirs.ply

Note that these are the Gaussians, before fitting of edges

The points sampled on the curves These results are saved as regular ply files with prefex edge_sampled_points in the output directory of the corresponding scan.

Evaluation - ABC-NEF

You can use the file eval.py (taken largely from EMAP) to evaluate the results. This has been verified for ABC-NEF dataset. You can use pre_compute_ABC_gt_sampled.py to precompute sampled points on the GT edges. This can also be done for individual scans with the option --save_sampled_points when running fit_edges.py.

As discussed in the paper, evaluation over the pseudo-groundtruth on the DTU dataset is not recommended. However, this can be done by changing the appropriate output paths in EMAP's evlaution script for DTU.

Notes

The evaluation numbers can have minute variation from those in the paper due to small config changes and the stochasiticity in the clustering process.

Improvements/TODOs

Robustness: The code was developed inspired from Nerfstudio's Nerfacto, while not building upon one of the existing 3DGS pipelines, to have more control over the pipeline. In this process, several components, especially related to the adaptive control of the number and density of Gaussians, are not quite robust, especially as the scene grows larger. It would therefore be better to now use an existing Gaussian Splatting pipeline, like 3DGS-mcmc or Nerfstudio's splatfacto and integrate the components into it.
Post-processing of Edges: Merging of edges, joining close to junctions - Currently, merging and joining is not performed on the resulting edges and this can easily improve the edges produced - especially for object level datasets.
Custom dataset: This can be done using the colmap data parser. This has not been tested widely. You would need to extract edges using, PidiNet for example and provide the edge image path and appropriate image resolution scaling. An example will be added soon.
Logging - Need to add better prompts for failures and outputs.

Citing EdgeGaussians

If you have found the work useful or would like to compare against it, please cite it using the following:

@misc{chelani2024edgegaussians3dedge,
      title={EdgeGaussians -- 3D Edge Mapping via Gaussian Splatting}, 
      author={Kunal Chelani and Assia Benbihi and Torsten Sattler and Fredrik Kahl},
      year={2024},
      eprint={2409.12886},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2409.12886}, 
}

Contact

Please use the issues tab for any concerns. I would be happy to discuss and merge pull-requests if you notice a bug, or have any improvement ideas.

Acknowledgement

This project uses code fragments Nerfstudio's splatfacto and EMAP. We use pretrained DexiNed and PidiNet for edge map extraction. We thank all the authors for their great work and repos.

kunalchelani / EdgeGaussians

readme