search

SafeAILab / EAGLE

Official Implementation of EAGLE-1 (ICML'24) and EAGLE-2 (EMNLP'24)
https://arxiv.org/pdf/2406.16858
Apache License 2.0
780 stars 79 forks source link

ValueError: not enough values to unpack (expected 2, got 1) #4

Closed EnterpriseBCMe closed 9 months ago

EnterpriseBCMe commented 9 months ago

Hi，I encountered some problem when benchmarking throughputs. Here are the tracebacks

Traceback (most recent call last): File "/disk1/EAGLE/test_throughput.py", line 14, in output_ids=model.eagenerate(input_ids,temperature=0.5,max_new_tokens=512) File "/home/disk1/.conda/envs/EAGLE/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, kwargs) File "/disk1/EAGLE/model/ea_model.py", line 203, in eagenerate tree_logits, logits, hidden_state, sample_token = initialize_tree( File "/disk1/EAGLE/model/utils.py", line 164, in initialize_tree tree_logits, outputs, logits,hidden_state,sample_token = model( File "/home/disk1/.conda/envs/EAGLE/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, *kwargs) File "/disk1/EAGLE/model/ea_model.py", line 146, in forward ea_logits = self.ea_layer.topK_genrate(hidden_states,input_ids,self.base_model.lm_head,logits_processor) File "/home/disk1/.conda/envs/EAGLE/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(args, kwargs) File "/disk1/EAGLE/model/cnets.py", line 801, in topK_genrate out_hidden, past_key_values = self(hidden_states, input_ids=input_ids, use_cache=True) ValueError: not enough values to unpack (expected 2, got 1)

Maybe there is some bug within the forward? I've noticed that forward only return 1 value, hidden_states

Liyuhui-12 commented 9 months ago

Hello, a possible reason could be that you haven't set 

model.eval()

If the model is in training mode, the use_cache parameter will not work (as mentioned in line 573 of cnet.py, which is now commented out).

If the problem persists, could you provide the complete code that is causing the error?

EnterpriseBCMe commented 9 months ago

Hey, @Liyuhui-12 sorry for the late response, how could I forget the eval mode XD. It's exactly the reason and your explaination quite help me a lot to understand the code.