search

AlignmentResearch / tuned-lens

Tools for understanding how transformer predictions are built layer-by-layer
https://tuned-lens.readthedocs.io/en/latest/
MIT License
408 stars 42 forks source link

Fixed and simplified the CLI for the evaluation subcommand #108

Closed levmckinney closed 1 year ago

levmckinney commented 1 year ago

The CLI as it existed made it impossible to evaluate tuned lenses. This is resolved by this PR.

codecov[bot] commented 1 year ago

Codecov Report

Merging #108 (d9ff7f8) into main (9bf1f35) will increase coverage by 0.15%. The diff coverage is 85.71%.

:exclamation: Current head d9ff7f8 differs from pull request most recent head e2d23e2. Consider uploading reports for the commit e2d23e2 to get more accurate results

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #108      +/-   ##
==========================================
+ Coverage   80.95%   81.10%   +0.15%     
==========================================
  Files          32       32              
  Lines        2116     2112       -4     
==========================================
  Hits         1713     1713              
+ Misses        403      399       -4
Files Changed Coverage Δ
tests/scripts/test_integration.py 100.00% <ø> (ø)
tuned_lens/scripts/eval_loop.py 92.50% <85.71%> (+2.25%) :arrow_up: