Collecting "Future Improvements" for documentation

[rosinusserrano]

While working on this project I notice some stuff every now and then that will probably not be possible to accomplish in given time, but would be good to mention in our documentation to show that we at least have thought of it. Here is a short list of things I am thinking of, but Ill be adding more via comments. Feel free to do so as well.

1. Error handling in console: since this project is kind of a space for contribution there should be extensive error handling and explicit logging for why some things fail.
2. Better process updates in the UI: Show the user what exactly is happening in the background. Specially when downloading datasets or training models, since those processes may take quite a while
3. Validation: For example there should be tests whether the uploaded datasets are in the right format. Right now I just assume that there is a file called "transforms.json".

[rosinusserrano]

Clean up of temporary directories

[rosinusserrano]

Setup Tools for code quality (e.g. pre-commit hooks, linters, formatters)

[rosinusserrano]

use typescript for type safety and improved usability (e.g. autocompletion)

[rosinusserrano]

better management of env files, right now there is one at the root for supabase, and one in cloudnerf-backend

[rosinusserrano]

figure out dataset types in order to determine model-dataset compatibility (right now we simply define "compatibleModels" in the dataset inf.json)

[rosinusserrano]

Use asynchronous functions. When downloading the datasets for example, i launch the container using spawnSync, which launches a process in a synchronous manner, which allows me to easily run things after the download has finished. The problem is, that this blocks the server. Thus while downloading a dataset, the UI cant query exisitng datasets

[rosinusserrano]

the blender datasets fetched by nerfstudio dont contain the fl_x and fl_y fields in the transforms.json, because of which some models doesnt work on them. But one could easily create the transforms.json anew to contain these fields i think

[rosinusserrano]

prettier depiction of datasets: I am thinking of something like a grid view where one representative image of that dataset is shown with the name of the dataset as a subtitle

[rosinusserrano]

Disable selecting datasets that are not compatible with the selected model and vice versa

[rosinusserrano]

dont store everything in config js files

[rosinusserrano]

run models with different configs, such that one can compare the same model with different configs

[rosinusserrano]

use r3f for three js in react

[rosinusserrano]

keep track of running processes using by checking running containers.

E.g. when pressing the "run" button after having selected the dataset "dozer" and model "nerfacto", show a spinning icon in the UI while the container "nerfacto-dozer" exists

[rosinusserrano]

Check out related work (e.g. SDF studio)

[rosinusserrano]

store model checkpoint as intermediate step before storing results

[rosinusserrano]

add watchtower to automatically update docker images if available