Closed kaushikthedeveloper closed 10 months ago
@ChenMnZ , please note that have also included two PRs with the Llama-7b and Llama-13b quantized models as well, fixing the mlc-chat-config.json related issues. Tested post that it's working on A100 systems.
ChenMnZ/Llama-2-7b-chat-omniquant-w3a16g128asym This one solves the issue noted in Issue #18
Fixes changes to mlc-chat module based on the PR : https://github.com/mlc-ai/mlc-llm/issues/1080 , which changed
lib_path
argument tomodel_lib_path