add CKA as another distance metric - Githubissues

CalculatedContent / WeightWatcher

The WeightWatcher tool for predicting the accuracy of Deep Neural Networks

Apache License 2.0

1.47k stars 124 forks source link

add CKA as another distance metric #148

Open charlesmartin14 opened 2 years ago

charlesmartin14 commented 2 years ago

Ive been planning to implement CKA in weightwatcher Here's some info https://www.realclearscience.com/articles/2022/11/16/comparing_neural_networks_provides_a_look_into_the_black_box_of_artificial_intelligence_865086.html?utm_source=pocket_saves

https://haydn.fgl.dev/posts/a-better-index-of-similarity/

charlesmartin14 commented 1 year ago

This is now added in 0.6.5, however, this is only a standin The correct CKA method may require mean-centering the matrices

Will need to review in detail