Closed Linyou closed 1 year ago
Hi, @liruilong940607! Are you suggesting adding an option to the training script that allows switching between train and test modes? If yes, I think the test mode function should be a default option, just like its current state. Since it's closer to the original implementation and faster.
Oh sorry somehow I missed the change you made to the training script.
Changing it to default is reasonable, but in this case we need to carefully verify the metrics do not get affected noticeably comparing to the numbers on the website. -- We don't want future users get inconsistent metrics between the code and the website.
I'm happy to test it out in the weekend. But if you already have those numbers (PSNR & Test Time Diff) would be nice to mark them in this PR for future reference.
Hi, @liruilong940607. This is what I got using an RTX3090 GPU:
Train Mode Rendering
is the original rendering function
Scene | Chair | Drums | Ficus | Hotdog | Lego | Materials | Mic | Ship | Mean |
---|---|---|---|---|---|---|---|---|---|
PSNR | 35.70 | 25.09 | 33.75 | 37.45 | 35.73 | 29.56 | 36.60 | 30.39 | 33.03 |
Test Time | 11.46s | 16.51s | 14.89s | 17.38s | 13.42s | 18.01s | 12.13s | 28.91s | 16.59s |
Scene | Chair | Drums | Ficus | Hotdog | Lego | Materials | Mic | Ship | Mean |
---|---|---|---|---|---|---|---|---|---|
PSNR | 35.69 | 25.07 | 33.85 | 37.47 | 35.66 | 29.62 | 36.86 | 30.53 | 33.09 |
Test Time | 31.81s | 33.93s | 31.68s | 45.51s | 33.26s | 35.89s | 30.16s | 55.28s | 37.19s |
Scene | Bicycle | Garden | Bonsai | Counter | Kitchen | Room | Stump | Mean |
---|---|---|---|---|---|---|---|---|
PSNR | 22.38 | 24.49 | 30.17 | 26.86 | 27.92 | 30.50 | 22.31 | 26.38 |
Test Time | 14.97s | 7.12s | 6.05s | 5.33s | 3.90s | 5.77s | 10.88s | 7.72s |
Scene | Bicycle | Garden | Bonsai | Counter | Kitchen | Room | Stump | Mean |
---|---|---|---|---|---|---|---|---|
PSNR | 22.37 | 24.52 | 29.94 | 26.84 | 27.90 | 30.50 | 22.36 | 26.35 |
Test Time | 22.24s | 15.15s | 9.85s | 8.83s | 7.50s | 10.19s | 14.95s | 12.67s |
Please note that the Test Time
includes the computation of PSNR and is calculated by directly adding the timer to the code at here link
Awesome thanks for providing those numbers!! This looks great!! I'm gonna merge this PR then. Seems like the website do not need to update as the PSNR basically stays the same.
Very cool PR!
Thanks Linyou! Do you want to merge this PR in its current state or also adding the usage of this test mode function in the script? I’m happy with both options. Let me know when you done with this PR and want it to be merged.