search

qnguyen3 / chat-with-mlx

An all-in-one LLMs Chat UI for Apple Silicon Mac using MLX Framework.
https://twitter.com/stablequan
MIT License
1.45k stars 131 forks source link

Download different quantize Model, But dir is the Same #199

Open 100ZZ opened 1 month ago

100ZZ commented 1 month ago

Qwen2-7B-Instruct-8bit.yaml

original_repo: Qwen/Qwen2-7B-Instruct mlx-repo: mlx-community/Qwen2-7B-Instruct-8bit quantize: 8bit default_language: multi system_prompt: You are a helpful assistant

Qwen2-7B-Instruct-4bit.yaml

original_repo: Qwen/Qwen2-7B-Instruct mlx-repo: mlx-community/Qwen2-7B-Instruct-4bit quantize: 4bit default_language: multi system_prompt: You are a helpful assistant

when download from chat-with-mlx UI，model dir name both are chat_with_mlx/models/download/Qwen2-7B-Instruct can save model dir name to Qwen2-7B-Instruct-8bit and Qwen2-7B-Instruct-4bit ？ i can download different quantize model in different dir first