Closed mianqiu00 closed 3 months ago
Qqp
zero-shot
python inference.py -m Llama-2-7b-hf -d qqp -shots 0 --ranking_type prob
Our results: 63.17. (all no)
python inference.py -m Meta-Llama-3-8B -d qqp -shots 0 --ranking_type prob
Our results: 62.53.
one-shot
python inference.py -m Llama-2-7b-hf -d qqp -shots 1 --ranking_type prob
Our results: 36.83. (all yes)
python inference.py -m Meta-Llama-3-8B -d qqp -shots 1 --ranking_type prob
Our results: 69.96.
32-shot
python inference.py -m Llama-2-7b-hf -d qqp -shots 32 --ranking_type prob
Our results: 54.83.
python inference.py -m Meta-Llama-3-8B -d qqp -shots 32 --ranking_type prob
Our results: 68.50.
llama2 with zero-shot or one-shot predict all no or yes, this problem haven't been solved by adjusting instruction and ranking_type
Qqp
zero-shot
Our results: 63.17. (all no)
Our results: 62.53.
one-shot
Our results: 36.83. (all yes)
Our results: 69.96.
32-shot
Our results: 54.83.
Our results: 68.50.
llama2 with zero-shot or one-shot predict all no or yes, this problem haven't been solved by adjusting instruction and ranking_type