search

huggingface / text-generation-inference

Large Language Model Text Generation Inference
http://hf.co/docs/text-generation-inference
Apache License 2.0
8.86k stars 1.04k forks source link

ValueError: Unsupported model type llava #1962

Closed pseudotensor closed 3 months ago

pseudotensor commented 4 months ago

System Info

2.0.4 docker image

Information

Tasks

Reproduction

https://github.com/huggingface/text-generation-inference/pull/1709#issuecomment-2134264274

Expected behavior

llava next 1.6 was supposedly added, but 34b fails.

72b can't use sharding: https://github.com/huggingface/text-generation-inference/pull/1709#issuecom

pseudotensor commented 4 months ago

Even lmms-lab/llama3-llava-next-8b fails same way

github-actions[bot] commented 3 months ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.