search

runpod-workers / worker-vllm

The RunPod worker template for serving our large language model endpoints. Powered by vLLM.
MIT License
213 stars 82 forks source link

Using mistral 0.3 #79

Closed Sapessii closed 3 weeks ago

Sapessii commented 2 months ago

Hi,

when launching it with "mistralai/Mistral-7B-Instruct-v0.3" I get the following error KeyError: 'layers.0.attention.wk.weight'

Do you know how to fix it?

Thank you