Use of NTK Condition Number in Combination with ZEN Score.

[SXKJames]

Hello MingLin,

I found your NAS approach to be very interesting.

Did you ever try to combine the ZEN-Score with the NTK-score like the TE-NAS paper did? The TE-NAS paper suggests that combining the NTK-Score with a expressivity measure would improve the overall performance.

Thanks, SXK

[MingLin-home]

Hi SXKJames,

Good idea! We did not try Zen + NTK. It might worth a try. Please be aware that the NTK also suffers from the BN normalization problem, so you should calibrate the gradient norm too.

Please let me know if it works!

[dovedx]

这是来自QQ邮箱的假期自动回复邮件。   邮件我已经成功接收，我会尽快处理并答复，谢谢！