search

ShenhanQian / GaussianAvatars

[CVPR 2024 Highlight] The official repo for "GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians"
https://shenhanqian.github.io/gaussian-avatars
Other
520 stars 84 forks source link

Discrepancy in net_type Option for `lpips()` between `train.py` and `metrics.py` #35

Open HoYoung-City opened 5 months ago

HoYoung-City commented 5 months ago

Hi! I noticed an inconsistency in the calculation of the Metric in train.py. The lpips() function is used without specifying the net_type option, as seen here: https://github.com/ShenhanQian/GaussianAvatars/blob/f649a11bae59a1622500345b21641781f01fbab7/train.py#L296

Then, the default net_type for 'lpips()' is set to 'alex': https://github.com/ShenhanQian/GaussianAvatars/blob/f649a11bae59a1622500345b21641781f01fbab7/lpipsPyTorch/__init__.py#L8

On the other hand, in metrics.py, the net_type for lpips() is set to 'vgg': https://github.com/ShenhanQian/GaussianAvatars/blob/f649a11bae59a1622500345b21641781f01fbab7/metrics.py#L74

This discrepancy could explain why there are differences in the values. I believe that using 'alex' might be the more appropriate option, as it is closer to the results presented in the paper.

Could you please confirm this?

Thank you for looking into this.

ShenhanQian commented 5 months ago

Hi, thanks for raising this question!

The reported numbers in our paper came from the evaluation in train.py, which means we used alexnet for LPIPS.

Meanwhile, we didn't use metrics.py but kept it in the repo, thinking it might be helpful in some use cases. Therefore, there can be some inconsistency.

BTW, did you get consistent numerical results from metric.py as those from train.py after moving to alexnet?

HoYoung-City commented 4 months ago

I used alexnet in metric.py , but I got different numerical results.

I haven't found the exact cause yet. It could be my mistake.

Thank you for your reply.