how about kd trianing without ema?

[DL-Practise]

thanks for your paper and code. my question is how about the model performance when i not use the eam option. it means i didn't pass the option "--use_ema"

[bokyeong1015]

Hi,

In the below test with our code, we didn't observe notable differences in

• (1) training with vs. without the --use_ema option, and
• (2) evaluating the unet folder vs. the unet_ema folder, when training with --use_ema .

Whether to use the EMA checkpoint or not seems quite debatable [discussion link].

---

Setup

• Model: SD-v1.5 + BK-SDM-Small
• Training Batch Size 128 (=32x4 where 4 for gradient_accumulation_steps)
• Evaluation on full MS-COCO 30K benchmark

Visual Results (50000-th iteration)

Quantitative Results

• FID Score (lower is better)	FID↓	train with --use_ema	with --use_ema	without --use_ema
  iteration	eval with unet_ema	unet	unet
  5000	18.63	18.57	19.24
  10000	17.65	18.36	17.44
  20000	16.23	16.65	16.66
  30000	16.41	16.98	16.79
  40000	16.26	16.23	16.84
  50000	16.50	17.34	16.72

• Inception Score (higher is better)	IS↑	train with --use_ema	with --use_ema	without --use_ema
  iteration	eval with unet_ema	unet	unet
  5000	25.04	25.55	24.53
  10000	27.64	27.13	27.13
  20000	29.14	28.59	28.59
  30000	30.19	29.48	29.75
  40000	30.49	30.01	29.78
  50000	30.71	30.84	30.49

• CLIP Score (higher is better)

  CLIP↑	train with --use_ema	with --use_ema	without --use_ema
  iteration	eval with unet_ema	unet	unet
  5000	0.2447	0.2446	0.2439
  10000	0.2557	0.2546	0.2546
  20000	0.2630	0.2622	0.2620
  30000	0.2655	0.2627	0.2648
  40000	0.2674	0.2647	0.2653
  50000	0.2681	0.2659	0.2659