Hi, I am following the instructions to run the synthetic benchmark.
I use the LLaMA-2-chat-hf model, and I specify the path in run.sh
GPUS="1" # GPU size for tensor_parallel.
ROOT_DIR="RULER/results" # the path that stores generated task samples and model predictions.
MODEL_DIR="RULER/models" # the path that contains individual model folders from HUggingface.
ENGINE_DIR="" # the path that contains individual engine folders from TensorRT-LLM.
However, I found that the prepare.py takes so long to run
Hi, I am following the instructions to run the synthetic benchmark.
I use the LLaMA-2-chat-hf model, and I specify the path in
run.sh
However, I found that the
prepare.py
takes so long to runAnd the evaluation score is always 0.0, for example, I use the
hotpotQA
benchmark, and it keeps outputting:I'd appreciate any help on this.