junxnone / aiwiki

AI Wiki

https://junxnone.github.io/aiwiki

17 stars 2 forks source link

ML AutoML NAS Metrics #394

Open junxnone opened 3 years ago

junxnone commented 3 years ago

NAS Metrics

Accuracy
Efficiency
- GPU Days = N X D = Number of GPUs X Number of days
Kendall Tau metric

Kendall Tau metric
NC	concordant
ND	discordant
τ = 1	两个 `rankings` 一致, 算法在搜索和评估阶段都表现比较好
τ = -1	两个 `rankings` 完全相反, 算法搜索表现好, 但是评估阶段差
τ = 0	两个 `rankings` 不相关, 类似随机搜索

junxnone commented 3 years ago

junxnone/tech-io#805