File "/root/miniconda3/lib/python3.10/site-packages/vllm/worker/worker.py", line 249, in execute_model
output = self.model_runner.execute_model(seq_group_metadata_list,
File "/root/miniconda3/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/root/miniconda3/lib/python3.10/site-packages/vllm/worker/model_runner.py", line 790, in execute_model
self.set_active_loras(lora_requests, lora_mapping)
File "/root/miniconda3/lib/python3.10/site-packages/vllm/worker/model_runner.py", line 901, in set_active_loras
self.lora_manager.set_active_loras(lora_requests, lora_mapping)
File "/root/miniconda3/lib/python3.10/site-packages/vllm/lora/worker_manager.py", line 113, in set_active_loras
self._apply_loras(lora_requests)
File "/root/miniconda3/lib/python3.10/site-packages/vllm/lora/worker_manager.py", line 235, in _apply_loras
self.add_lora(lora)
File "/root/miniconda3/lib/python3.10/site-packages/vllm/lora/worker_manager.py", line 250, in add_lora
self._lora_manager.activate_lora(lora_request.lora_int_id)
File "/root/miniconda3/lib/python3.10/site-packages/vllm/lora/models.py", line 615, in activate_lora
result = super().activate_lora(lora_id)
File "/root/miniconda3/lib/python3.10/site-packages/vllm/lora/models.py", line 355, in activate_lora
module.set_lora(index, module_lora.lora_a, module_lora.lora_b,
File "/root/miniconda3/lib/python3.10/site-packages/vllm/lora/layers.py", line 800, in set_lora
lora_b = self.slice_lora_b(lora_b)
File "/root/miniconda3/lib/python3.10/site-packages/vllm/lora/layers.py", line 786, in slice_lora_b
lora_b = [lora_b_q, lora_b_k, lora_b_v]
UnboundLocalError: local variable 'lora_b_k' referenced before assignment
Neither of lora_b_q, lora_b_k, lora_b_v has a default value, unfortunately my adapter only applied to q and v at the time, then the error occured, I guess.
Your current environment
🐛 Describe the bug
After installed
0.4.2
using pip:Then I tried to start a service using entrypoint as usual.
I got the error when I make a request:
Then check the code of tag
v0.4.2
, I found this:https://github.com/vllm-project/vllm/blob/c7f2cf2b7f67bce5842fedfdba508440fe257375/vllm/lora/layers.py#L773-L787
Neither of
lora_b_q, lora_b_k, lora_b_v
has a default value, unfortunately my adapter only applied to q and v at the time, then the error occured, I guess.