Open ilongshan opened 1 year ago
This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 60 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.
Hope it can be added
This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 60 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.
Hope it can be added
This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 60 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.
Hope it can be added
This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 60 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.
Comment to keep it open
This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 60 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.
Comment to keep it open
Library name
TinyChatEngine
Library description
TinyChatEngine: On-Device LLM Inference Library
Source repository URL
https://github.com/mit-han-lab/TinyChatEngine
Project homepage (if different from the source repository)
https://mit-han-lab.github.io/TinyChatEngine/
Anything else that is useful to know when adding (such as optional features the library may have that should be included)
Running large language models (LLMs) on the edge is useful: running copilot services (coding, office, smart reply) on laptops, in-car entertainment systems, vision-language assistants in robots or spaceships. Users can get instant responses without transmitting any local data to the internet, addressing privacy concerns, as sensitive data remains localized and reduces the risk of potential breaches.
TinyChat project consists of LLM model compression algorithms and TinyChatEngine, the inference engine that implements the compressed LLM targeting edge devices.