search

colorful-liyu / 3DQD

[CVPR'23] Official PyTorch implementation of 3DQD: Generalized Deep 3D Shape Prior via Part-Discretized Diffusion Process
78 stars 8 forks source link

There is a bug in the evaluation code #15

Open Negai-98 opened 1 year ago

Negai-98 commented 1 year ago

In 'uncond_metrics.py' at line 41, the 'pairwise' in methods like LION, PF, PVD, etc., is centered around the test samples, whereas in your code, it is centered around the generated samples. To align with other methods for comparison, you should modify: M_rs_cd,M_rs_emd = _pairwise_EMD_CD_(sample_pcs, ref_pcs, batch_size, EMD_flag, verbose=True) to M_rs_cd, M_rs_emd = _pairwise_EMD_CD_(ref_pcs, sample_pcs, batch_size, EMD_flag, verbose=True) This might be the reason for the notable performance difference in the supplementary materials section of your paper compared to other papers. Additionally, could you provide generated samples for the unconditional generation task or share the corrected evaluation results for quantitative comparison with your method?

colorful-liyu commented 1 year ago

Thank you for your carefulness. I checked the _pairwise_EMD_CD_() in PVD official code just now in https://github.com/alexzhou907/PVD/blob/main/metrics/evaluation_metrics.py. I am actually confused, the variable names they used in the definition and invocation of _pairwise_EMD_CD_ are different. I will remeasure the metrics in a few days.