elevenlabs integration error

[zag2art]

I try to use your demo with elevenlabs.

    const talkResponse = await fetch(`${DID_API.url}/talks/streams/${streamId}`,
      {
        method: 'POST',
        headers: { Authorization: `Basic ${DID_API.key}`, 'Content-Type': 'application/json' },
        body: JSON.stringify({
          "script": {
            "type": "text",
            "subtitles": "false",
            "provider": {
              "type": "elevenlabs",
              "voice_id": "BMKpAZGNzkYvXm2hPSuP"
            },
            "input": "Hello, Niraj, my name is B K. Thank you for your help with this clone"
          },
          'driver_url': 'bank://lively/',
          'config': {
            'stitch': true,
          },
          'session_id': sessionId
        })
      });

but it generates an error

POST https://api.d-id.com/talks/streams/strm_PNmi1SRve84D4DLroBJ_2

payload

{
    "script": {
        "type": "text",
        "subtitles": "false",
        "provider": {
            "type": "elevenlabs",
            "voice_id": "BMKpAZGNzkYvXm2hPSuP"
        },
        "input": "Hello, Niraj, my name is B K. Thank you for your help with this clone!"
    },
    "driver_url": "bank://lively/",
    "config": {
        "stitch": true
    },
    "session_id": "AWSALB=Q5Gyd67+7D7aO/jGklXdslfpmIP5cWV+7hu6COtUKrcJ0lWMy+FYISrkSvK3EuFQV2fhLKAr1dD3wR2ekeW/ARpoLJ0bm/Y0BiZFxlwIlCFXzRWzcW/21KCSmh7P; Expires=Wed, 24 May 2023 13:57:14 GMT; Path=/; AWSALBCORS=Q5Gyd67+7D7aO/jGklXdslfpmIP5cWV+7hu6COtUKrcJ0lWMy+FYISrkSvK3EuFQV2fhLKAr1dD3wR2ekeW/ARpoLJ0bm/Y0BiZFxlwIlCFXzRWzcW/21KCSmh7P; Expires=Wed, 24 May 2023 13:57:14 GMT; Path=/; SameSite=None; Secure"
}

response status code - 400

{
    "kind": "TextToSpeechProviderError",
    "description": "SSML is only supported by Microsoft Azure",
    "details": {
        "provider": "elevenlabs"
    }
}

What is wrong?

[orgoro]

please send script.ssml: false with your request

[zag2art]

this is also doesn't work

{
    "script": {
        "type": "text",
        "subtitles": "false",
        "provider": {
            "type": "elevenlabs",
            "voice_id": "BMKpAZGNzkYvXm2hPSuP"
        },
        "input": "Hello, Niraj, this is B K. Don't you think my clone is awesome?",
        "ssml": false
    },
    "config": {
        "stitch": true
    },
    "session_id": "AWSALB=8psTxAIKNnqijpKaaLcLsv3PUW3jas+eZDHtL6fGDB4NpSP5XvXzWyN67NFhNuhHT+hPyJv5fXiZcO8PgdrYHu7LNxI3CfB3Mb6zTjyUlrp13Zc/5brpjUjZ6p4j; Expires=Thu, 25 May 2023 10:28:45 GMT; Path=/; AWSALBCORS=8psTxAIKNnqijpKaaLcLsv3PUW3jas+eZDHtL6fGDB4NpSP5XvXzWyN67NFhNuhHT+hPyJv5fXiZcO8PgdrYHu7LNxI3CfB3Mb6zTjyUlrp13Zc/5brpjUjZ6p4j; Expires=Thu, 25 May 2023 10:28:45 GMT; Path=/; SameSite=None; Secure"
}

Can you please re open the issue

[orgoro]

What error are you getting with ssml : false

[zag2art]

the same error

{
    "kind": "TextToSpeechProviderError",
    "description": "SSML is only supported by Microsoft Azure",
    "details": {
        "provider": "elevenlabs"
    }
}

>curl 'https://api.d-id.com/talks/streams/strm_izzkJV9w3xePYJUOK1oJ3' \                                                                                                                             16:50:21
>   -H 'authority: api.d-id.com' \
>   -H 'accept: */*' \
>   -H 'accept-language: en-US,en;q=0.9,ru;q=0.8' \
>   -H 'authorization: Basic bW................................................................C' \
>   -H 'cache-control: no-cache' \
>   -H 'content-type: application/json' \
>   -H 'dnt: 1' \
>   -H 'origin: http://localhost:4000' \
>   -H 'pragma: no-cache' \
>   -H 'referer: http://localhost:4000/' \
>   -H 'sec-ch-ua: "Google Chrome";v="113", "Chromium";v="113", "Not-A.Brand";v="24"' \
>   -H 'sec-ch-ua-mobile: ?0' \
>   -H 'sec-ch-ua-platform: "macOS"' \
>   -H 'sec-fetch-dest: empty' \
>   -H 'sec-fetch-mode: cors' \
>   -H 'sec-fetch-site: cross-site' \
>   -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36' \
>   --data-raw $'{"script":{"type":"text","subtitles":"false","provider":{"type":"elevenlabs","voice_id":"BMKpAZGNzkYvXm2hPSuP"},"input":"Hello, Niraj, this is B K. Don\'t you think my clone is awesome?","ssml":false},"config":{
"stitch":true},"session_id":"AWSALB=v/NrY9O2uQTiocZNdgIEpHhCser2VIrMgCp8T2jULAwFHIdE0BtNyIzJJU3kk/mWJ8NtPzVPeF8HRgks2lp35kfd+ytFt2OyDmX02w9oOjBnRLqTKSXovFpwSS7E; Expires=Thu, 25 May 2023 13:14:40 GMT; Path=/; AWSALBCORS=v/NrY9O2uQTiocZN
dgIEpHhCser2VIrMgCp8T2jULAwFHIdE0BtNyIzJJU3kk/mWJ8NtPzVPeF8HRgks2lp35kfd+ytFt2OyDmX02w9oOjBnRLqTKSXovFpwSS7E; Expires=Thu, 25 May 2023 13:14:40 GMT; Path=/; SameSite=None; Secure"}' \
>   --compressed
{"kind":"TextToSpeechProviderError","description":"SSML is only supported by Microsoft Azure","details":{"provider":"elevenlabs"}}

[rmw362]

Yes, this issue should not be closed.....you can't use any other TTS provider for the streaming endpoint right now. Perhaps you should have waited to release this feature and charge for it until it was actually fully tested.

[rmw362]

@orgoro any updates on this? The API docs are misleading if only microsoft TTS works for the streaming endpoint

[Khosraw]

@orgoro, Any updates on this? Is the streaming API compatible with ElevenLabs at the moment?

[orgoro]

Yes :)

You can see it in the docs or use the implementation under studio.d-id.com/agents