elevenlabs / elevenlabs-js

The official JavaScript (Node) library for ElevenLabs Text to Speech.
https://elevenlabs.io
MIT License
60 stars 7 forks source link

Streaming using elevenlabs.generate is not working #39

Open DeeRock94 opened 1 month ago

DeeRock94 commented 1 month ago

Following your recent documentation on the README, it states streaming generated audio can be done via the below:

image

I've attempted to achieve this with the code below

    const audioStream = await elevenlabs.generate({
        stream: true,
        voice: "Liam",
        text: "This is a... streaming voice",
        model_id: "eleven_multilingual_v2"
    });

    stream(audioStream)

This doesn't seem to work, now when I am using play() and playing it directly, with the stream: false set (or not defined), it works fine, however the second I go to stream it, it seems to throw an error in my node application:

node:internal/process/promises:288
            triggerUncaughtException(err, true /* fromPromise */);
            ^

[UnhandledPromiseRejection: This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason "null".] {
  code: 'ERR_UNHANDLED_REJECTION'
}

Node.js v18.12.1

Any suggestions? I've attempted to try and catch this error but I can't seem to find anything of value from it.

Update:

Okay so I've managed to get a more specific error than just "null"

node:internal/process/promises:288
            triggerUncaughtException(err, true /* fromPromise */);
            ^

Error: write EPIPE
    at afterWriteDispatched (node:internal/stream_base_commons:160:15)
    at writeGeneric (node:internal/stream_base_commons:151:3)
    at Socket._writeGeneric (node:net:917:11)
    at Socket._write (node:net:929:8)
    at writeOrBuffer (node:internal/streams/writable:392:12)
    at _write (node:internal/streams/writable:333:10)
    at Writable.write (node:internal/streams/writable:337:10)
    at D:\Development\ai-assistant\node_modules\elevenlabs\wrapper\stream.js:42:74
    at Generator.next (<anonymous>)
    at fulfilled (D:\Development\ai-assistant\node_modules\elevenlabs\wrapper\stream.js:5:58) {
  errno: -4047,
  code: 'EPIPE',
  syscall: 'write'
}
dsinghvi commented 1 month ago

Hey @DeeRock94 -- we'll follow up here. Thanks for filing this issue!

DeeRock94 commented 1 month ago

HI just an update to this, I finally managed to catch the culprit, it took 3 runs and on the 3rd I just got lucky and it decided to actually tel lme what the error was. It was due to not having MPV (https://mpv.io/) installed. I'm developing this on a Windows (Regretting not using my mac now) with hope to move this to a Linux based OS in the near future.

It seems ElevenLabs might be using MPV under the hood for the stream() method. After installing MPV and updating my PATH I managed to get this working.

dsinghvi commented 1 month ago

That's right, awesome!