search

Riga2 / TensoSDF

[SIGGRAPH 2024] TensoSDF: Roughness-aware Tensorial Representation for Robust Geometry and Material Reconstruction
MIT License
46 stars 3 forks source link

Very low GPU utilization #1

Closed SuLvXiangXin closed 2 weeks ago

SuLvXiangXin commented 4 weeks ago

Thank you for your wonderful work! But I have a little confusion about the training speed.

In the paper, it said 4 hours + 1.5 hours on RTX4090. However, I test the compressor scene on RTX A6000. For the first geometry stage, it takes 8+ hours to complete the training (18w iters), and the GPU utilization is only about 50%. For the next material stage, it takes also about 8 hours to complete the training (10w iters), and the GPU utilization is only about 15%, which I think mainly due to the data preparation in every iteration.

Riga2 commented 3 weeks ago

Thank you for your wonderful work! But I have a little confusion about the training speed.

In the paper, it said 4 hours + 1.5 hours on RTX4090. However, I test the compressor scene on RTX A6000. For the first geometry stage, it takes 8+ hours to complete the training (18w iters), and the GPU utilization is only about 50%. For the next material stage, it takes also about 8 hours to complete the training (10w iters), and the GPU utilization is only about 15%, which I think mainly due to the data preparation in every iteration.

Hi, sorry for the late reply. Thanks for your interests on our project! I have tested the compressor scene again on my RTX 4090.

The training time and GPU utilization on the geometry stage are shown below. shape_training_time shape_training It seems normal on my machine. The GPU utilization is around 85-90%.

And the training time and GPU utilization on the material stage are shown below. mat_training_time mat_training The GPU utilization is around 35-40%, but the training time is still around 1.5 hrs.

Hope it can help you!

SuLvXiangXin commented 3 weeks ago

Thanks for your reply!