search

ziyc / drivestudio

A 3DGS framework for omni urban scene reconstruction and simulation.
https://ziyc.github.io/omnire/
MIT License
437 stars 29 forks source link

parallel trainging #23

Open zhangSaneC opened 3 days ago

zhangSaneC commented 3 days ago

I have a 16G GPU, and in training process, cuda out of memory error occur.

May I ask how to do parallel training like using two GPUs in this project?

Thanks for your reply.

ziyc commented 2 days ago

Hi @zhangSaneC, parallel training of Gaussian Splatting on multi-GPUs didn't exist until this new work came out. We haven't implemented support for it yet.

Typically, training for large outdoor scenes requires more GPU memory due to the millions of Gaussians needed for fitting. You can limit the growth speed of Gaussians by increasing the Gaussian densification threshold here: https://github.com/ziyc/drivestudio/blob/388642a6a998833cb388e0f1a65dfc2071cc6a61/configs/omnire.yaml#L60

Be careful though, as this will affect the final reconstruction quality. You may need to balance between computational load and final quality.