Suggestions asking for inputting data

[RoveFlemington]

Are there any special suggestions for inputting burst images, such as should the camera movement/tremor avoid being too large?? Actually, in my testing, significant optical flow changes make my reconstructed background image blurry.

[RoveFlemington]

[Ilya-Muromets]

Absolutely, I kind of go over this in the supplemental section of the paper: https://light.princeton.edu/wp-content/uploads/2023/12/neural-spline-fields.pdf

But the gist of it is that NSF is a very simple stochastic optimization model, you're kind of relying on it falling into a local optima (separating the occlusion from the transmission) based on millions of small updates. If that optimum is too far away from the initialization (e.g., your camera moves a lot), then there's a good chance the model will never find it.

As a rule of thumb, it seems to do fairly well when the motion is on the scale of the occlusion (e.g., if you have a fence that's X pixels wide, then during the burst it should ideally move between X and 3X pixels up/down/left/right to be well reconstructed).

[RoveFlemington]

Thanks a lot for your useful comment. I have another question here, that is, when I train the model, the GLIBC version of the server cannot match to the tiny-cuda-nn. Is it convenient for you to directly give {occlusion}'s weight data? I'll appreciate it much! My e-mail: zhangzhe21@tsinghua.org.cn

[Ilya-Muromets]

I'm afraid you'll need to have tiny-cuda-nn working even to load the weights / test inference. Have you tried a prebuilt wheel (e.g., https://github.com/OutofAi/tiny-cuda-nn-wheels) ?