CyberAgentAILab / layout-dm

LayoutDM: Discrete Diffusion Model for Controllable Layout Generation [Inoue+, CVPR2023]
https://cyberagentailab.github.io/layout-dm
Apache License 2.0
211 stars 23 forks source link

Regarding to the align metric. #23

Closed Sierkinhane closed 1 year ago

Sierkinhane commented 1 year ago

Hi, I used the code to reproduce the results and I got:

alignment-ACLayoutGAN: 0.0098 (15.5840%)        
alignment-LayoutGAN++: 0.0015 (26.8312%)        
alignment-NDN: 0.0834 (5.9249%)        
average_iou-BLT: 0.0669 (0.8978%)        
average_iou-VTN: 0.1921 (1.3181%)
        coverage: 0.4248 (1.6913%)        
        density: 0.5789 (3.0516%)        
        fid: 6.4562 (6.0454%)        
        overlap-ACLayoutGAN: 10.7990 (1.5706%)        
        overlap-LayoutGAN: 0.3682 (2.5028%)
        overlap-LayoutGAN++: 0.8676 (1.2573%)
        precision: 0.7337 (1.0037%)        recall: 0.9008 (1.0541%)

I would like to ask which one is the Align metric used in Table 2?

naoto0804 commented 1 year ago

(though this is answered on an email, but I just copy&paste here for reference) Thank you for your interest in our work! For overlap and alignment, I usually use LayoutGAN++'s metrics. We scale the values of Alignment by 100× for visibility. Both are mentioned in appendix (B.5).

billy322 commented 7 months ago

Hi, for the Overlap metrics, you are using the LayoutGAN++ metrics, but the Overlap in the LayoutGAN++ paper fall into a scale between 0-1000? E.g., 188.77±0.93 and 32.40±0.89 in Table 3 in the LayoutGAN++ paper [1]. Can I know how to convert to that scale for reporting? Thanks.

[1] https://dl.acm.org/doi/pdf/10.1145/3474085.3475497

naoto0804 commented 7 months ago

@billy322: please refer to #33