search

liuff19 / Semantic-Ray

[CVPR 2023] Semantic Ray: Learning a Generalizable Semantic Field with Cross-Reprojection Attention
https://liuff19.github.io/S-Ray/
MIT License
78 stars 2 forks source link

The reconstruction quality on ScanNet dataset. #3

Closed lifuguan closed 1 year ago

lifuguan commented 1 year ago

Hi, I have trained the model following the instruction python run_training.py --cfg configs/cra/train_cra_scannet.yaml. However, the PSNR metric is much lower than the paper reported. Here's my validation results:

{
"loss_rgb_nr": [0.04487277, 0.07840069, 0.05396397, 0.059577584, 0.033189584, 0.081671566, 0.051437616, 0.055169743, 0.024891067, 0.03389036], 
"loss_rgb_nr_fine": [0.047508813, 0.08060137, 0.052870948, 0.056064654, 0.03304764, 0.083196364, 0.050065003, 0.052776337, 0.024463316, 0.033602122], 
"loss_semantic": [0.5113131, 0.5086871, 0.69861794, 0.78223276, 0.47776088, 0.8893344, 0.33830953, 0.89945805, 0.5300619, 0.68642217], 
"psnr_nr": [18.680141, 14.5373, 16.678677, 16.937946, 18.557934, 14.577075, 17.545328, 18.21395, 18.189692, 18.35741], 
"ssim_nr": [0.47412324, 0.47946468, 0.50667745, 0.5182882, 0.40986508, 0.4980549, 0.63974226, 0.63125074, 0.5714904, 0.4930398], 
"psnr_nr_fine": [18.790533, 14.478284, 16.743114, 16.868551, 18.565668, 14.662598, 17.602137, 18.225819, 18.223934, 18.400364], 
"ssim_nr_fine": [0.47979182, 0.48036927, 0.50494874, 0.5174515, 0.4187762, 0.49905854, 0.6366769, 0.62079793, 0.57648945, 0.49423438], 
"miou": [0.52838165, 0.42462546, 0.36073428, 0.31907237, 0.3830865, 0.2829054, 0.51568216, 0.4454773, 0.5013875, 0.29229742], 
"total_accuracy": [0.65284127, 0.7044097, 0.5975464, 0.4774825, 0.7020239, 0.52696943, 0.8264945, 0.4289473, 0.6657821, 0.612379], 
"class_average_accuracy": [0.6912788, 0.5571608, 0.50963336, 0.47534674, 0.49139518, 0.4168798, 0.574, 0.64264566, 0.5889166, 0.42346135]}

Is the hyper-params goes wrong?

LinShan-Bin commented 1 year ago
  1. The model doesn't seem to converge. Did you train enough steps?
  2. Did you load the pretrained neuray model? Many photos in the scannet dataset have ghosting and black edges which are bad for the training process. Therefore, we larverage neuray for better rendering quality.
lifuguan commented 1 year ago

Thanks for reply. I'll double check the configure file.