search

OpenGVLab / OmniQuant

[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.
MIT License
663 stars 50 forks source link

Fix ChatModule initalization with model_lib_path argument #28

Closed kaushikthedeveloper closed 10 months ago

kaushikthedeveloper commented 10 months ago

Fixes changes to mlc-chat module based on the PR : https://github.com/mlc-ai/mlc-llm/issues/1080 , which changed lib_path argument to model_lib_path

kaushikthedeveloper commented 10 months ago

@ChenMnZ , please note that have also included two PRs with the Llama-7b and Llama-13b quantized models as well, fixing the mlc-chat-config.json related issues. Tested post that it's working on A100 systems.