This repository contains the code for 3D shape fitting to predicted surface normals, as shown in our paper:

FOUND: Foot Optimisation with Uncertain Normals for Surface Deformation using Synthetic Data \ Winter Conference on Applications of Computer Vision 2024 \ Oliver Boyne, Gwangbin Bae, James Charles, and Roberto Cipolla \ [arXiv] [project page]

Quickstart

1) git clone --recurse-submodules http://github.com/OllieBoyne/FOUND 2) Install dependencies: pip install -r requirements.txt 3) Download the pretrained FIND model to data/find_nfap 4) Download our benchmark foot dataset to data/scans 5) Fit a single scan:

python FOUND/fit.py --exp_name <exp_name> --data_folder <data_folder>

You can use --cfg <file>.yaml to use a config file to set parameters. See args.py for all arguments, and example-cfg.yaml for an example config file.

6) Evaluate all of our reconstruction dataset:

python FOUND/eval.py --data_folder <data_folder> --gpus <gpu_indices>

gpu_indices is a space separated list, e.g. --gpus 0 1 2 3

Data

We provide our synthetic foot dataset, SynFoot, which contains 50K synthetic foot scans, with RGB, normals, and masks.

We also provide a benchmark multiview evaluative dataset, Foot3D.

Related work

Please check out all of our projects that built into this work!

• FIND - Generative foot model
• Surface Normal Estimation w/ Uncertainty
• BlenderSynth - Synthetic data generation

Citation

If you use our work, please cite:

@inproceedings{boyne2024found,
            title={FOUND: {F}oot {O}ptimisation with {U}ncertain {N}ormals for Surface {D}eformation using Synthetic Data},
            author={Boyne, Oliver and Bae, Gwangbin and Charles, James and Cipolla, Roberto},
            booktitle={Winter Conference on Applications of Computer Vision (WACV)},
            year={2024}
}

Troubleshooting

If you have any issues with trimesh and shapely, see misc/shapely.md.