[New Model]: Mistral-Nemo

[Hambaobao]

The model to consider.

https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407

The closest model vllm already supports.

• https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/llama.py
• https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/mixtral.py

What's your difficulty of supporting the model you want?

No response

[jonzhep]

we just need to pull head_dim from config i think https://github.com/huggingface/transformers/commit/4c040aba02b0283619a06bdc40ecf868508b9e52

[simon-mo]

Closed by #6548. It will be released next week. Please try build from source for now. Thank you!