Error in sps inference - Githubissues

zankner commented 1 month ago

Running:

python -m evaluation.inference_sps --drafter-path double7/vicuna-68m --model-path lmsys/vicuna-7b-v1.3 --model-id sps_1 --bench-name spec_bench --temperature 0.0 --dtype "float16"

I get the error:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/mnt/workdisk/zack/Spec-Bench/evaluation/inference_sps.py", line 125, in <module>
    run_eval(
  File "/mnt/workdisk/zack/Spec-Bench/evaluation/eval.py", line 54, in run_eval
    get_answers_func(
  File "/usr/lib/python3/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/mnt/workdisk/zack/Spec-Bench/evaluation/eval.py", line 111, in get_model_answers
    output_ids, new_token, step, accept_length_tree = forward_func(
  File "/mnt/workdisk/zack/Spec-Bench/evaluation/inference_sps.py", line 19, in sps_forward
    output_ids, idx, accept_length_list = model.generate(
ValueError: not enough values to unpack (expected 3, got 1)

hemingkx commented 1 month ago

We re-ran the codes and did not encounter this error. Could you check if your local code contains this line?

hemingkx commented 1 month ago

This issue was closed because it has been inactive for 7 days. If there are any other questions, please open a new issue or send me an email.

hemingkx / Spec-Bench

Error in sps inference #9