Closed the-crypt-keeper closed 1 year ago
Use same GGUF format that was used to eval v1
Completed at Q4_K_M and Q5_K_M.
Use same GGUF format that was used to eval v1
94 opened to track pulling up AWQ and vLLM to latest and trying that out with this model