search

octoml / mlc-llm

Enable everyone to develop, optimize and deploy AI models natively on everyone's devices.
https://mlc.ai/mlc-llm
Apache License 2.0
5 stars 8 forks source link

Merge with latest `mlc-ai/main` (`5bc3ffa6f682a4cf42fdeba3a4c505d0e7c08c3c`) #253

Closed sunggg closed 5 months ago

sunggg commented 5 months ago

This PR pulls the latest changes in the upstream. One of the key change is the renaming of mlc_chat to mlc_llm as they deprecate the legacy flow in pre-SLM era. Once this is merged, I will follow-up with the ollm changes.

During resolving the merge conflicts, I found that there are a couple of subtle differences in the quantization flow and presharding. I believe I currently kept the existing behavior, but it would be great if @csullivan @vinx13 @JosephTheOctonaut can confirm.

For the review, it is safe to ignore other directories outside of python/.

sunggg commented 5 months ago

@vinx13, thank you for the review. I think I reflected your comments, would you mind taking another look? Appreciate it!