tts exception - Githubissues

Self Checks

[X] This is only for bug report, if you would like to ask a question, please head to Discussions.
[X] I have searched for existing issues search for existing issues, including closed ones.
[X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
[X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.8.3

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

tts exception

use xinference depoly ChatTTS

2024-09-27 04:26:36,718 xinference.api.restful_api 1 ERROR    [address=0.0.0.0:40435, pid=1056] 'Chat' object has no attribute 'gpt'
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/xinference/api/restful_api.py", line 1376, in create_speech
    out = await model.speech(
  File "/usr/local/lib/python3.10/dist-packages/xoscar/backends/context.py", line 231, in send
    return self._process_result_message(result)
  File "/usr/local/lib/python3.10/dist-packages/xoscar/backends/context.py", line 102, in _process_result_message
    raise message.as_instanceof_cause()
  File "/usr/local/lib/python3.10/dist-packages/xoscar/backends/pool.py", line 656, in send
    result = await self._run_coro(message.message_id, coro)
  File "/usr/local/lib/python3.10/dist-packages/xoscar/backends/pool.py", line 367, in _run_coro
    return await coro
  File "/usr/local/lib/python3.10/dist-packages/xoscar/api.py", line 384, in __on_receive__
    return await super().__on_receive__(message)  # type: ignore
  File "xoscar/core.pyx", line 558, in __on_receive__
    raise ex
  File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.__on_receive__
    async with self._lock:
  File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.__on_receive__
    with debug_async_timeout('actor_lock_timeout',
  File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive__
    result = await result
  File "/usr/local/lib/python3.10/dist-packages/xinference/core/utils.py", line 45, in wrapped
    ret = await func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/xinference/core/model.py", line 90, in wrapped_func
    ret = await fn(self, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/xoscar/api.py", line 462, in _wrapper
    r = await func(self, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/xinference/core/model.py", line 670, in speech
    return await self._call_wrapper_binary(
  File "/usr/local/lib/python3.10/dist-packages/xinference/core/model.py", line 401, in _call_wrapper_binary
    return await self._call_wrapper("binary", fn, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/xinference/core/model.py", line 114, in _async_wrapper
    return await fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/xinference/core/model.py", line 415, in _call_wrapper
    ret = await asyncio.to_thread(fn, *args, **kwargs)
  File "/usr/lib/python3.10/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.10/dist-packages/xinference/model/audio/chattts.py", line 73, in speech
    rnd_spk_emb = self._model.sample_random_speaker()
  File "/usr/local/lib/python3.10/dist-packages/ChatTTS/core.py", line 162, in sample_random_speaker
    return self._encode_spk_emb(self._sample_random_speaker())
  File "/usr/local/lib/python3.10/dist-packages/ChatTTS/core.py", line 182, in _sample_random_speaker
    dim: int = self.gpt.gpt.layers[0].mlp.gate_proj.in_features
AttributeError: [address=0.0.0.0:40435, pid=1056] 'Chat' object has no attribute 'gpt'

✔️ Expected Behavior

none exception

❌ Actual Behavior

No response

I found a similar discussion that might be relevant to your issue: Xinference failed to access the local model, which is still open and was last updated on 2024-08-13 ^[1].

To resolve the 'Chat' object has no attribute 'gpt' exception when using xinference to deploy ChatTTS in a self-hosted Docker environment, follow these steps:

Install Xinference:
```
pip install "xinference[all]"
```
Start Xinference Locally:
```
xinference-local
```
This will start Xinference with the default endpoint http://127.0.0.1:9997. If you need to allow access from any non-local client, you can configure it with -H 0.0.0.0.
Deploy the Model:
- Visit http://127.0.0.1:9997 in your browser.
- Select and deploy the model you need, such as baichuan-chat.
Obtain the Model UID:
- Copy the model ID from the Running Models page, e.g., 2c886330-8849-11ee-9518-43b0b8f40bea.
Configure Dify to Use the Deployed Model:
- Go to Settings > Model Providers > Xinference in Dify.
- Fill in the model information:
  - Model Name: The name of the deployed model, e.g., baichuan-chat.
  - Server URL: The endpoint address of Xinference, e.g., http://<Machine_IP>:9997 (replace <Machine_IP> with your machine's IP address).
  - Model UID: The UID obtained from the Running Models page.
Ensure Correct Network Configuration:
- If using Docker, ensure that the Dify container can access the Xinference endpoint. The Dify container cannot access localhost inside; you need to use the host IP address.

By following these steps and ensuring proper network configuration, you should be able to resolve the 'Chat' object has no attribute 'gpt' exception ^[2]^[3]^[4].

To continue talking to Dosu, mention @dosu.

langgenius / dify