premAI-io / prem-services

AI Services Maintained by Prem Core Team.
MIT License
6 stars 8 forks source link

Mistral-7B_q5, mac M1/M2 binaries #117

Closed filopedraz closed 1 year ago

filopedraz commented 1 year ago
{
  "weightsDirectoryUrl": "https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF/resolve/main/",
  "weightsFiles" : ["mistral-7b-instruct-v0.1.Q5_0.gguf"],
  "binariesUrl": {
    "aarch64-apple-darwin": "https://github.com/premAI-io/prem-services/releases/download/v1/cht-llama-cpp-mistral-1-aarch64-apple-darwin"
  },
  "serveCommand": "./cht-llama-cpp-mistral-1-aarch64-apple-darwin --model_path=mistral-7b-instruct-v0.1.Q5_0.gguf"
},
{
  "weightsDirectoryUrl": "https://huggingface.co/TheBloke/Mistral-7B-OpenOrca-GGUF/resolve/main/",
  "weightsFiles" : ["mistral-7b-openorca.Q5_K_S.gguf"],
  "binariesUrl": {
    "aarch64-apple-darwin": "https://github.com/premAI-io/prem-services/releases/download/v1/cht-llama-cpp-mistral-1-aarch64-apple-darwin"
  },
  "serveCommand": "./cht-llama-cpp-mistral-1-aarch64-apple-darwin --model_path=mistral-7b-openorca.Q5_K_S.gguf"
}