search

rbalestr-lab / stable-SSL

https://rbalestr-lab.github.io/stable-SSL.github.io/dev/
MIT License
4 stars 2 forks source link

[Feature Proposal/Discussion] Training Dynamics Utilities #101

Closed vipulSharma18 closed 1 week ago

vipulSharma18 commented 1 week ago

Discussed in https://github.com/rbalestr-lab/stable-SSL/discussions/100

Originally posted by **vipulSharma18** November 14, 2024 Have different training dynamics utils as modular utilities in stable-SSL. Example features: 1. Rank Me style performance prediction. 2. Propagation of signal like utilities. (Correlation in positive and negative pairs across layers) 3. KNN clustering for embeddings Any other interesting training dynamics measures like for compositionality in latent space, or dynamics of attention scores across layers, circuit probing and evolution throughout training.
vipulSharma18 commented 1 week ago

Might be irrelevant to stable-ssl at this stage.