search

unslothai / unsloth

Finetune Llama 3.2, Mistral, Phi, Qwen 2.5 & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
18.3k stars 1.28k forks source link

Unsloth这个对cogvlm2-llama3-chinese-chat-19B 推理加速这个会实现吗？ #521

Open gehong-coder opened 5 months ago

gehong-coder commented 5 months ago

model, tokenizer = get_model_tokenizer(model_type, torch.float16,load_in_4bit=True,model_kwargs={'device_map': 'auto'},use_unsloth=True NotImplementedError: Unsloth: .cache/modelscope/hub/ZhipuAI/cogvlm2-llama3-chinese-chat-19B not supported yet!

danielhanchen commented 5 months ago

Wait is this a vision model?