timaeus-research / devinterp

Tools for studying developmental interpretability in neural networks.
77 stars 14 forks source link

Will/Non-TPU dependent Integration with Aether #92

Closed wz-ml closed 3 months ago

wz-ml commented 3 months ago

Integrations with the Aether repository before the monorepo merge.

Additions:

svwingerden commented 3 months ago

After some bugfixes, all tests (except acc/ord) now pass. But there are a few small issues:

  1. Please remove george's sampling method code, and add it into its own separate branch (no pr needed)
  2. Please update OnlineLLCEstimator to also take in both nbeta and temperature, like was done for LLCEstimator
  3. Please update the example notebooks to adjust for the made changes. (I've fixed introduction.ipynb and normal_crossing.ipynb already)
wz-ml commented 3 months ago

Check on changes 1, 2, and partly 3. I've updated all the notebooks except for the two mentioned and made minor fixes to callbacks. All of them should be working except for diagnostics.ipynb, which I've updated but is blocked on the default init_loss not being implemented for OnlineLLCEstimator. Merge at your leisure : )