Closed levmckinney closed 1 year ago
Merging #105 (2feccf7) into main (c9921f5) will increase coverage by
0.06%
. The diff coverage is82.60%
.
@@ Coverage Diff @@
## main #105 +/- ##
==========================================
+ Coverage 80.88% 80.95% +0.06%
==========================================
Files 32 32
Lines 2077 2116 +39
==========================================
+ Hits 1680 1713 +33
- Misses 397 403 +6
Impacted Files | Coverage Δ | |
---|---|---|
tuned_lens/plotting/trajectory_plotting.py | 84.94% <42.85%> (-3.43%) |
:arrow_down: |
tuned_lens/plotting/prediction_trajectory.py | 91.87% <77.77%> (-0.48%) |
:arrow_down: |
tests/plotting/test_prediction_trajectory.py | 100.00% <100.00%> (ø) |
This PR adds support to for rank as a metric that can be visualized using a prediction trajectory. Rank in this case is the rank of the target token relative to all the other tokens in the predictions at each layer in the prediction trajectory. This can be accessed using the new
rank
method. In addition, this PR adds support for a log scaling the metrics when visualizing a specificTrajectoryStatistic
.