[New Model]: facebook/seamless-m4t-v2-large - Githubissues

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

https://docs.vllm.ai

Apache License 2.0

26.03k stars 3.81k forks source link

[New Model]: facebook/seamless-m4t-v2-large #6017

Open frittentheke opened 2 months ago

frittentheke commented 2 months ago

The model to consider.

https://huggingface.co/facebook/seamless-m4t-v2-large

The closest model vllm already supports.

No response

What's your difficulty of supporting the model you want?

The yet unsupported architecture SeamlessM4Tv2Model.

Their own inference code: https://github.com/facebookresearch/seamless_communication
HF code: https://github.com/facebookresearch/seamless_communication/tree/main/docs/m4t

But the SeamlessM4Tv2Model does require encoders (https://github.com/huggingface/transformers/blob/e65502951593a76844e872fee9c56b805598538a/src/transformers/models/seamless_m4t_v2/modeling_seamless_m4t_v2.py#L4319) - Likely this is tackled with https://github.com/vllm-project/vllm/issues/187 ?

Micla-SHL commented 1 month ago

Hi, I recently noticed this model: https://huggingface.co/facebook/seamless-m4t-v2-large.

My question: documnent: https://docs.vllm.ai/en/latest/models/adding_model.html
Does this document also include guidelines for unsupported model architectures?