Closed dtch1997 closed 3 months ago
[!WARNING]
Review failed
The pull request is closed.
This update enhances the greater_than.ipynb
notebook by introducing new setup and error-handling procedures, along with significant restructuring. Key improvements involve splitting functionality into explicit functions within the EAP-IG model's ecosystem, incorporating caching and ablation capabilities, refining graph node and edge management, and expanding testing coverage. Overall, these changes boost the robustness, usability, and extensibility of the EAP-IG model and its associated components.
Files/Modules | Change Summary |
---|---|
examples/eap_ig/greater_than.ipynb | Added setup section, renamed sections, added error handling, updated code execution count, added new functions and cell |
sae_eap/attribute.py | Modified get_cache_hook default parameter, updated hook assignments for caching, added assertion, moved score addition logic |
sae_eap/graph/build.py | Renamed node construction functions, added build_attn_nodes , refactored build_layer_attn_nodes |
sae_eap/prune.py | Introduced new pruning classes and methods, added utility functions, enhanced edge/node pruning logic |
sae_eap/ablate.py | Added functions for edge ablation hooks, subgraph checks, and ablation input settings |
sae_eap/cache.py | Added caching tensor functionalities, CacheDict , and cache setter hooks |
sae_eap/core/types.py | Renamed hook types, added TLForwardHookFn , TLBackwardHookFn , ForwardHook , and BackwardHook |
sae_eap/data/handler.py | BatchHandler.get_logits method accepts InputType literal, added __repr__ to SinglePromptHandler |
sae_eap/data/ioi.py | Introduced make_ioi_single function |
sae_eap/graph/init.py | Added TensorNode , SrcNode , DestNode , and TensorEdge classes |
sae_eap/graph/edge.py | TensorEdge got new properties src and dest |
sae_eap/graph/graph.py | Added has_node and has_edge methods |
sae_eap/graph/node.py | Enhanced activation handling methods in TensorNode and other node types |
sae_eap/runner.py | Introduced run_attribution and run_ablation functions |
tests/integration/attribute/test_attribute.py | Removed test_attribute function |
.../runner/test_run_ablate.py | Added functions for pruner pipeline creation and ablation testing |
.../runner/test_run_attribute.py | Introduced test_attribute function with attribution process testing |
Mid the nodes and edges thrives a graph anew,
Pruned and cached, its circuits true.
EAP-IG stands tall, in CUDA's embrace,
Handling errors with a quartz-like grace.
With hooks and tensors, pruners align,
In the realm of models, they brightly shine.
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?
Roadmap
Summary by CodeRabbit
New Features
Enhancements
Bug Fixes
Documentation