apartresearch / deepdecipher

🦠 DeepDecipher: An open source API to MLP neurons
https://apartresearch.com
MIT License
9 stars 0 forks source link

Improve neuron similarity algorithm #99

Open albertsgarde opened 1 year ago

albertsgarde commented 1 year ago

The neuron similarity algorithm was based on very basic knowledge of what N2G was and should be improved. This should wait until the N2G package is ready so it can rely on the graph itself instead of just the activating and important tokens. The threshold should also be rethought. At the moment any neurons with similarity above a certain threshold are included, but this leaves some neurons with no similar neurons and some with very many. Perhaps instead take the top n similar neurons?