search

minghanqin / LangSplat

Official implementation of the paper "LangSplat: 3D Language Gaussian Splatting" [CVPR2024 Highlight]
https://langsplat.github.io/
Other
674 stars 72 forks source link

Inconsistency between reported metric and self-trained results #60

Open seanzhuh opened 3 months ago

seanzhuh commented 3 months ago

Hi, I've traiend on waldo_kitchen of LERF dataset using your code but the reproduced result (Loc. Acc is 0.8182) was lower than that reported in your paper (0.955). What could be the reason?

garrisonz commented 3 days ago

More result for reference.

I train and eval on teatime of LERF dataset follow README instruction, using the default parameter.

  self-trained in paper
mIoU scores 58.0 65.1
Localization accuracy 83.1 88.1
AbdallahElsayed73 commented 2 hours ago

I personally trained on Teatime dataset and I got 88.1 localization accuracy. The only difference I found with self-training was indeed the evaluation on Waldo Kitchen although I got 77.3% localization accuracy. I also got accuracy of 67.6% on ramen dataset while it was reported to be 73.2% in the paper