search

Asparagus15 / GaussianShader

code for GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces
Other
298 stars 25 forks source link

Paper Result #17

Open scy04 opened 7 months ago

scy04 commented 7 months ago

Can you provide a full instruction on ShinyBlender dataset? I have no idea how to reproduce the paper result. And I am wondering why the data in your paper are inconsistent. 111 222

scy04 commented 7 months ago

@Asparagus15 Your reply will be highly appreciated

TangZJ commented 6 months ago

I'm also having issues replicating the results. But i think that figure 1(a) result is based on per frame.

ingra14m commented 6 months ago

Here are my results using the latest commit. I used the command python train.py -s path/to/refnerf --eval -m output/refnerf/name -w --brdf_dim 0 --sh_degree -1 --lambda_predicted_normal 2e-1 --brdf_env 512 under 800x800 resolution for all the experiments.

Ref-NeRF

<!DOCTYPE html> Scene PSNR SSIM LPIPS(VGG)
ball 29.27 0.9563 0.1428
car 28.44 0.939 0.0475
coffee 31.02 0.9689 0.0855
helmet 28.13 0.9516 0.0902
teapot 43.58 0.9957 0.0107
toaster 23.93 0.9107 0.1024
Mean 30.73 0.954 0.0798

img_v3_028t_77c44c99-388d-4919-8a74-9750ef8bc4bg img_v3_028t_a53c42fc-2c86-4d0e-8fa0-09ef91a0ab1g

NSVF

<!DOCTYPE html> Scene PSNR SSIM LPIPS (VGG)
Bike 37.38 0.9917 0.0066
Lifestyle 27.36 0.9636 0.0509
Palace 36.55 0.9791 0.0198
Robot 37.00 0.9938 0.0082
Spaceship 32.61 0.9847 0.0158
Steamtrain 35.27 0.9903 0.0103
Toad 34.50 0.9795 0.0227
Wineholder 30.16 0.9657 0.0291
Average 33.85 0.981 0.0204

tbh, I don't think the quality of the shortest axis normal is sufficient for good reflection. The absence of GS's geometry makes it difficult to distinguish between environment and material through geometry (normal). I've even used the gt normal of refnerf to train models, but the improvement in geometry hardly helps the 3D-GS rendering, and the reflective part is still blurred.

Under the premise that differentiable surface rendering currently cannot surpass volumetric rendering, I truly believe that able-nerf is an astonishing method, except for being too slow (4 V100s for 3 days) and the interpretability of color tokens (maybe the backpack language model could help?) @TangZJ .