Open ill13 opened 1 month ago
Looks like you're using the quantized models, it might be hampering the model's performance on numerical data. I cannot replicate this issue on the official meta llama models, I get 11110 from both 8b and 70b models.
Try increasing the temperature, 0.01 sounds quite low.
The issue is the model is consistently modifying my numeric input.
I'm not looking for the model to do simple math, I'm looking for the model to not fuzz my input.
Trying What is 7777 + 3333? and both models keep changing my input to "77,777 + 33,333".
Both models will often give the correct answer to 77,777 + 33,333, however that's not what was asked.
This concern was confirmed by another user on Reddit.
Running: