Are there any parameters that are related with the number of input frames?

[jyj0121]

Hello, I'm really appreciated for your nice work and also sharing your valuable code. I'm trying to train HexPlane model with about 32 frames' multi-view video (like nv3d dataset), but while training I only got a black blank image corresponding to the validation view. (It works fine with the nv3d dataset and my dataset with 300 frames). Should I control some parameters(maybe time_grid_init/time_grid_final? ) according to the number of input frames? Thank you very much.

[Caoang327]

Hi. Thanks for the appreciation. If it could work well on your dataset with 300 frames, you may want to change the time_grid_init and time_grid_final to work with 32 frames. When there are only 32 time frames, while the time_grid is bigger, then there would be many time frames that cannot get access to learning signals. You can try to set time_grid_init = 8 (0.25 * frame numbers) and time_grid_final = 31 or 15. If it works, feel free to tune it. Let me know if it works or not.

[JunhyeongDoyle]

Hello,

I wanted to say thank you for your input on this issue. The comments and responses here have been extremely useful in helping me to successfully train a 100-frame video. I have made modifications to some parts of the neural_3D_dataset_NDC.py code (such as tensor sizes, timestamps, etc.) and tweaked the 'time_grid' parameter in the config file as suggested in the comments. It's now working perfectly, and I appreciate the information shared here.

On a related note, I have another question. Currently, the hexplane model renders at the same frame count as the training data. I was wondering if it would be possible to adjust this to render at an arbitrary frame count of my choosing, and if so, where I would need to make such modifications?

Thank you for sharing your work :)

[Caoang327]

Hi. Yes you can render at arbitrary frame count as you choice. You just need to have a set of timesteps corresponding to these frame count (ranging from -1 to 1) and pass them to hexplanes.

[ZcsrenlongZ]

Hi! Will you relese your trained models in Plenoptic dataset? Thanks!