Added the 13b model, meta-llama/Llama-2-13b-chat-hf, to see if I get as good TruthfulQA boost as on Llama-2-7b-chat-hf and the rest of 7b models. Alas no 😞
Did a parameter sweep of alpha (15 20 25 30) and K (48 64 80 96) for the Llama-2-70b-chat-hf to see if some will beat the alpha=0 baseline. No luck either 😢
Documented my work
One reason I wanted yo look at the 13b is that it fits on disk and in memory of one A100, just as 7b, to eliminate possible bug sources in the 70b. But it looks like 13b does not get the True*Info boost either ...
I've also spot-checked alpha=5 on 70b and 13b - no good there too.
meta-llama/Llama-2-13b-chat-hf, to see if I get as goodTruthfulQAboost as onLlama-2-7b-chat-hfand the rest of 7b models. Alas no 😞alpha(15 20 25 30) andK(48 64 80 96) for theLlama-2-70b-chat-hfto see if some will beat thealpha=0baseline. No luck either 😢One reason I wanted yo look at the 13b is that it fits on disk and in memory of one A100, just as 7b, to eliminate possible bug sources in the 70b. But it looks like 13b does not get the
True*Infoboost either ...I've also spot-checked
alpha=5on 70b and 13b - no good there too.