search

camel-ai / camel

🐫 CAMEL: Finding the Scaling Law of Agents. The first and the best multi-agent framework. https://www.camel-ai.org
https://docs.camel-ai.org/
Apache License 2.0
5.59k stars 679 forks source link

[Feature Request] Add `SmolLM` model and WebLLM #850

Open lightaime opened 2 months ago

lightaime commented 2 months ago

Required prerequisites

Motivation

Add SmolLM https://huggingface.co/blog/smollm, https://huggingface.co/spaces/HuggingFaceTB/SmolLM-360M-Instruct-WebGPU. And WebGPU support.

https://webllm.mlc.ai/

Solution

No response

Alternatives

No response

Additional context

No response

Wendong-Fan commented 2 months ago

Hey @lightaime , this model is supported by ollama, should we do native integration? refer: https://ollama.com/library/smollm

koch3092 commented 2 months ago

The difference between WebLLM and LLM Web APP:

Note: Ignore browser dependency on GPU

image

koch3092 commented 2 months ago

Here are the key features of WebLLM :

  1. WebLLM leverages the WebGPU on the user's local machine for hardware acceleration, enabling high-performance language model inference directly in the browser. This removes server dependencies, reduces costs, enhances privacy and personalization, all while lowering operational expenses**.
  2. WebLLM natively supports a wide range of popular models, including Llama, Hermes, Phi, Gemma, RedPajama, Mistral, SmolLM and Qwen**, making it adaptable for various tasks.
  3. It is fully compatible with the OpenAI API, offering features like JSON mode, function calling, and streaming output, simplifying integration for developers.
  4. WebLLM allows the integration of custom models in MLC format to meet specific needs, enhancing flexibility in model deployment.
  5. As a standalone package, WebLLM can be quickly integrated into projects via NPM, Yarn, or CDN, and comes with comprehensive examples and a modular design that makes it easy to connect with UI components.
  6. It supports streaming output and real-time interaction, making it suitable for applications like chatbots and virtual assistants.
  7. By delegating computational tasks to Web Workers or Service Workers, WebLLM optimizes UI performance and efficiently manages the lifecycle of models.
  8. It provides examples for building Chrome extensions, allowing users to extend browser functionalities with ease.