Intelli-IoT

Framework & Library

• PipeLine

  pipe = pipeline("text-classification")
  pipe("This restaurant is awesome")

• NVIDIA TAO

Service

• Langchain
• 3rd party API

Model light scheme

• AutoGPTQ - https://github.com/PanQiWei/AutoGPTQ
• GPTQ - https://github.com/IST-DASLab/gptq
• Gpt4all - https://github.com/nomic-ai/gpt4all

Edge Model

• Distil-Whisper
• Whisper-Edge
  • Whisper 성능 - https://www.clien.net/service/board/lecture/17699124
• MS Phi-1.5

Cloud service

• Clova
• PeriFlow

APIs

• OpenAI API
• HyperClovaX API

Etc

PTQ for LLM
8bit 이면 LLM 추론에 충분하다

• OpenAI chatGPT 음성 기능 https://www.aitimes.com/news/articleView.html?idxno=155407
• stable diffusion video model https://news.hada.io/topic?id=11971
• MS SLM 오르카2 https://www.aitimes.com/news/articleView.html?idxno=155371
• Meta video gen / edit model https://news.hada.io/topic?id=11952
• Google Music gen model Lyria https://news.hada.io/topic?id=11918