Closed krzz2q closed 1 month ago
Thank you for your feedback. I have also reproduced the issue and will fix it shortly
You can add a --prefix_caching True
flag to temperately solve the problem while waiting for us to fix it
You can add a
--prefix_caching True
flag to temperately solve the problem while waiting for us to fix it OK. Thank you.
+1 encountered the same problem, but I used Generation mode, so --prefix_caching True
is not valid for me
Description:
I encountered an
IndexError
while running the inference script for the Meta-Llama-3-8B-Instruct model using the command below:The error traceback is as follows:
Steps to Reproduce:
evaluate()
function.Expected Behavior:
The evaluation should complete without errors.
Environment:
Additional Information:
It seems the error is related to the
batch_sampler.py
file, where the code attempts to access an index that is out of range. Any help or guidance on how to resolve this issue would be greatly appreciated. If additional logs or information is needed, please let me know.Thank you!