Closed AlpinDale closed 1 month ago
Very simple implementation, doesn't work with TP.
Example model here: https://huggingface.co/alpindale/Mistral-7B-Instruct-v0.2-EETQ
Very simple implementation, doesn't work with TP.
Example model here: https://huggingface.co/alpindale/Mistral-7B-Instruct-v0.2-EETQ