Week 3 - Discussion [2.5.23 - 6.5.23]

[git-bauerseb]

We have now a couple of choices when it comes to training inverse rendering models. Thus, I'd suggest that we have a consistent naming of models and data folders in order not to mess things up.

I'd propose the following:

Datasets

All data is stored under /mnt/hdd/data. Right now, Simon provided 3 head datasets which I would name the following:

• head_01: The initial set of 16 images
• head_38|85: The follow up datasets with depth/normal maps

Thus, /mnt/hdd/data/head_38 would contain the unzipped data of the second head he provided. If we preprocess the dataset in order to use it for some application, then prefixing the dataset with the name of the application would be good. E.g. when using the NVIDIA pipeline for the second head then store everything in /mnt/hdd/data/nvdiffrec_head_38. Also, update the ABOUT.md script on what steps you performed to transform the input dataset into the desired format. If you need to write/need to use some scripts (e.g. colmap2sdfstudio.py) then store them in the scripts folder (/mnt/hdd/scripts) and comment them thoroughly s.t. everyone knows what they do.

Models

All models should be stored in /mnt/hdd/models. The ABOUT.md contains the command used to train the models (like in the dataset case).

Please prefix the name of the model with the method you used to train it and the version of your model (e.g. (nerfstudio|sdfstudio)_(neus|neus_facto|nerf)_head_(01|38|85))_version_(00xx). If it's the 139th model you trained using sdfstudio and neus-facto on the last head then use: sdfstudio_neus_facto_head_85_version_0139

The current directory structure is not valid, I'll try to correct it according to the above scheme.

Of course we could discuss a better naming scheme, but I think the above is reasonable in order not to mess up directory structure or delete models others have trained.

EDIT: If you trained models/ created datasets by yourself, then you can name them anything you want. The only thing is to document them and make sure, everyone can reproduce the results. If you encountered any errors and needed to install a package/use a script, please write it in the ABOUT.md in the respective folders.

[git-bauerseb]

sdfstudio

sdfstudio works on the provided datasets but it does not work when converting the custom nerfstudio dataset to sdfstudio format. It shows mono_depth_path KeyError in the meta_data.json file.

[git-bauerseb]

This is how it looks like when I use sdfstudio with the following settings:

ns-train neus-facto \
    --output-dir /mnt/hdd/models/sdfstudio_neusfacto_head_01_version_0001 \
    --pipeline.model.sdf-field.inside-outside True \
    --pipeline.model.near-plane 0.05 \
    --pipeline.model.far-plane 4.0 \
    --vis viewer \
    nerfstudio-data \
    --scene-scale 0.6 \
    --auto-scale-poses True \
    --center-poses True \
    --data /mnt/hdd/data/nerfstudio_head_01

I don't see a way to move the bounding box, so I'll write a script to change translation in cam2world matrices.