search

0hq / WebGPT

Run GPT model on the browser with WebGPU. An implementation of GPT inference in less than ~1500 lines of vanilla Javascript.
https://kmeans.org
Other
3.61k stars 206 forks source link

what different with mlc-ai/web-llm #21

Closed novohool closed 1 year ago

novohool commented 1 year ago

https://github.com/mlc-ai/web-llm

cnrpman commented 1 year ago

Same question. Also WebGPT is usually referring this paper in the field of LLM: https://arxiv.org/abs/2112.09332v3 Hope clarifying the project name decision

0hq commented 1 year ago

Sure.

WebLLM is using TVM and compiling from C++ to WebAssembly, which is more performant but vastly more complicated. This is built to be both a solid educational reference as well as a place to start more complex projects from, being only written in a small amount of Javascript.

Considering using Javascript here has minor performance impacts vs. the simplicity it provides (mostly all computation is done on WebGPU, meaning Javascript isn't involved), this is preferable in various circumstances.

novohool commented 1 year ago

Sure.

WebLLM is using TVM and compiling from C++ to WebAssembly, which is more performant but vastly more complicated. This is built to be both a solid educational reference as well as a place to start more complex projects from, being only written in a small amount of Javascript.

Considering using Javascript here has minor performance impacts vs. the simplicity it provides (mostly all computation is done on WebGPU, meaning Javascript isn't involved), this is preferable in various circumstances.

the same as this proj? https://xenova.github.io/transformers.js

cnrpman commented 1 year ago

@jingslunt It's using onnxruntimeweb, mature but no webGPU backend for now.

novohool commented 1 year ago

@jingslunt It's using onnxruntimeweb, mature but no webGPU backend for now. webgpu merged https://github.com/microsoft/onnxruntime/pull/14579