search

xenova / transformers.js

State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!
https://huggingface.co/docs/transformers.js
Apache License 2.0
10.87k stars 658 forks source link

Support Conversational Task in transformers.js #146

Open HyunjunA opened 1 year ago

HyunjunA commented 1 year ago

Name of the feature In general, the feature you want added should be supported by HuggingFace's transformers library:

I request the task: conversational å Reason for request Why is it important that we add this feature? What is your intended use case? Remember, we are more likely to add support for models/pipelines/tasks that are popular (e.g., many downloads), or contain functionality that does not exist (e.g., new input type).

The addition of the Conversational task to the Hugging Face transformers.js library is important for several reasons. Firstly, it addresses the need for enhanced privacy in conversational AI applications. By allowing users to utilize Language Models (LLMs) in the front-end without sharing their OpenAI key or any private information, the Conversational task ensures that users can engage in conversations while safeguarding their privacy. This is crucial for users who are reluctant to disclose sensitive data and enables the application of LLMs in various sensitive areas.

Secondly, enabling the execution of the ChatGPT model in the front-end extends its capabilities to information-sensitive fields such as the medical domain and other domains where data privacy is critical. My intended use case for this ChatGPT model, which can run in the front-end, is as an AI data science assistant on Electronic Health Records (EHR). I plan to train LLMs for research purposes on EHR data and make them work in the front-end environment without sending sensitive patient health information to third-party servers. This allows for the development of real-time applications that leverage conversational models to provide valuable insights and assistance while maintaining data privacy and security. The Conversational task empowers developers to build conversational AI systems that adhere to strict data privacy requirements and can be used in sensitive domains where privacy is of utmost importance.

Furthermore, integrating the Conversational task into the Hugging Face transformers.js library expands its functionality and provides developers with a privacy-focused solution for implementing conversational AI applications. By leveraging the library's rich ecosystem of pre-trained models and tools, developers can create sophisticated conversational systems that respect user privacy, deliver seamless user experiences, and cater to a wide range of industries and domains.

Additional context The addition of the Conversational task to the Hugging Face transformers.js library not only addresses the demand for privacy-conscious conversational AI but also contributes to the development of ethical and secure systems. It enables developers to build innovative conversational applications that prioritize user privacy, empower users to have control over their data, and ensure secure interactions. This feature is valuable across various industries, including healthcare, customer service, education, and more, where privacy and data protection are paramount. By supporting the Conversational task, the Hugging Face transformers.js library becomes a versatile and privacy-enhanced platform for developing cutting-edge conversational AI solutions. Add any other context or screenshots about the feature request here.

xenova commented 1 year ago

This is definitely something we plan to add! Do you know which model you are planning to use? If so, we could try prioritize adding support for it. The main limiting factor for feasibly running these very large models in the browser is the lack of WebGPU support, which we are working on. And once it is supported, this will be a much greater priority.

HyunjunA commented 1 year ago

Thank you for your prompt response. We will utilize the Llama-65B and Alpaca-30B models.

bil-ash commented 1 year ago

Consider adding support for bloomz . It supports multilingual(56 languages) generation on all models and even multilingual prompt on 7.1B and 176B models (I guess those 2 models will only be usable on server and not in browser).

Also, the smallest model 560m will very easily run in the browser after quantixation and you do not have to wait till webgpu support