search

HeyGen-Official / StreamingAvatar

76 stars 54 forks source link

readme

Streaming Avatar Demo

NEW: We have an SDK now! The SDK is an NPM package that you can easily add to your website and use for Streaming Avatar functionality. Please check it out at https://www.npmjs.com/package/@heygen/streaming-avatar

We also have an example React app demonstrating the SDK's functionality. That is located at: https://github.com/HeyGen-Official/StreamingAvatarTSDemo

Introduction

This HeyGen Streaming Avatar demo is a starting point from which developers can adapt and build streaming sessions into their own websites and experiences.

Below, we have outlined a few questions that frequently pop up when users first try out this demo. Below that FAQ, you will find directions for installing and running this demo.

Getting Started FAQ

How do I get an API Key?

Either an an API Key or Trial Token from HeyGen is required to run this Streaming API demo. API Keys are reserved for Enterprise customers, whereas both Creator and Teams plan users can activate and use a Trial token. You can retrieve either the API Key or Trial Token by logging in to HeyGen and navigating to this page in your settings: https://app.heygen.com/settings?nav=API

Which Avatars can I use with this project?

By default, there are several Public Avatars that can be used in Streaming. (AKA Streaming Avatars.) You can find the Avatar IDs for these Public Avatars by navigating to app.heygen.com/streaming-avatar and clicking 'Select Avatar'.

In order to use a private Avatar created under your own account in Streaming, it must be upgraded to be a Streaming Avatar. Only 1. Finetune Instant Avatars and 2. Studio Avatars are able to be upgraded to Streaming Avatars. This upgrade is a one-time fee and can be purchased by navigating to app.heygen.com/streaming-avatar and clicking 'Select Avatar'.

Photo Avatars are not compatible with Streaming and cannot be used.

Which voices can I use?

Most of HeyGen's AI Voices can be used with the Streaming API. To find the Voice IDs that you can use, please use the List Voices v2 endpoint from HeyGen: https://docs.heygen.com/reference/list-voices-v2

Why am I encountering issues with testing?

Most likely, you are hitting your concurrent session limit. While testing the Streaming API, your account is limited to 3 concurrent sessions. Please endeavor to close unused Streaming sessions with the Close Session endpoint when they are no longer being used; they will automatically close after some minutes.

You can check how many active sessions you have open with the List Sessions endpoint: https://docs.heygen.com/reference/list-sessions

Installing the Demo

  1. Clone the repository.

    git clone https://github.com/HeyGen-Official/StreamingAvatar.git

  2. Open the index.js file and replace 'YourApiKey' with your API key:

    "apiKey": "YourApiKey";

  3. (optional) Open the server.js file and set your OpenAI API key to use talk mode:

    const openai = new OpenAI({
 apiKey: "<your openai api key>",
});

  4. open a terminal in the folder and then install the express and run the server.js:

    npm install express
node server.js

  5. you will see App is listening on port 3000!.

Using the Demo

  1. Open the web browser and enter the http://localhost:3000to start the demo.
  2. Click the "New" button to create a new session. The status updates will be displayed on the screen.
  3. After the session is created successfully, click the "Start" button to start streaming the avatar.
  4. To send a task to the avatar, type the text in the provided input field and click the "Repeat Text" button.
  5. In order to use Talk mode, set your OpenAI key in server.js before starting the server and click "Talk" button
  6. Once done, click the "Close Connection" button to close the session.

Remember, this is a demo and should be modified according to your needs and preferences. Happy coding!

Troubleshooting

In case you face any issues while running the demo or have any questions, feel free to raise an issue in this repository or contact our support team.

Please note, if you encounter a "Server Error", it could be due to the server being offline. In such cases, please contact the service provider.