dosubot[bot]
commented
1 month ago Hey @cycwll! I'm here to help you with any bugs, questions, or contributions you have for the Dify repository. Let's work together to solve any issues you're encountering.
To resolve the issue where the voice parameter is always empty when passed from Dify to xinference in your self-hosted Dify version 0.6.16 running on Docker, you need to ensure that the voice parameter is correctly handled and passed through the relevant functions and API calls.
From the provided code, it appears that the XinferenceModelExtraParameter class and related methods do not include a voice parameter. You will need to modify the code to include this parameter where necessary.
Here is an example of how you can modify the XinferenceModelExtraParameter class and the related methods to include the voice parameter:
- Update the
XinferenceModelExtraParameterclass to include thevoiceparameter:
class XinferenceModelExtraParameter:
model_format: str
model_handle_type: str
model_ability: list[str]
max_tokens: int = 512
context_length: int = 2048
support_function_call: bool = False
support_vision: bool = False
voice: str = "" # Add the voice parameter
def __init__(self, model_format: str, model_handle_type: str, model_ability: list[str],
support_function_call: bool, support_vision: bool, max_tokens: int, context_length: int, voice: str) -> None:
self.model_format = model_format
self.model_handle_type = model_handle_type
self.model_ability = model_ability
self.support_function_call = support_function_call
self.support_vision = support_vision
self.max_tokens = max_tokens
self.context_length = context_length
self.voice = voice # Initialize the voice parameter- Update the
_get_xinference_extra_parametermethod to extract thevoiceparameter from the response:
@staticmethod
def _get_xinference_extra_parameter(server_url: str, model_uid: str) -> XinferenceModelExtraParameter:
if not model_uid or not model_uid.strip() or not server_url or not server_url.strip():
raise RuntimeError('model_uid is empty')
url = str(URL(server_url) / 'v1' / 'models' / model_uid)
session = Session()
session.mount('http://', HTTPAdapter(max_retries=3))
session.mount('https://', HTTPAdapter(max_retries=3))
try:
response = session.get(url, timeout=10)
except (MissingSchema, ConnectionError, Timeout) as e:
raise RuntimeError(f'get xinference model extra parameter failed, url: {url}, error: {e}')
if response.status_code != 200:
raise RuntimeError(f'get xinference model extra parameter failed, status code: {response.status_code}, response: {response.text}')
response_json = response.json()
model_format = response_json.get('model_format', 'ggmlv3')
model_ability = response_json.get('model_ability', [])
voice = response_json.get('voice', '') # Extract the voice parameter
if response_json.get('model_type') == 'embedding':
model_handle_type = 'embedding'
elif model_format == 'ggmlv3' and 'chatglm' in response_json['model_name']:
model_handle_type = 'chatglm'
elif 'generate' in model_ability:
model_handle_type = 'generate'
elif 'chat' in model_ability:
model_handle_type = 'chat'
else:
raise NotImplementedError(f'xinference model handle type {model_handle_type} is not supported')
support_function_call = 'tools' in model_ability
support_vision = 'vision' in model_ability
max_tokens = response_json.get('max_tokens', 512)
context_length = response_json.get('context_length', 2048)
return XinferenceModelExtraParameter(
model_format=model_format,
model_handle_type=model_handle_type,
model_ability=model_ability,
support_function_call=support_function_call,
support_vision=support_vision,
max_tokens=max_tokens,
context_length=context_length,
voice=voice # Pass the voice parameter
)Additionally, ensure that your Dify Docker deployment is correctly configured to access the Xinference endpoint. According to the Dify documentation, you need to set the server URL to the machine's IP address instead of localhost:
使用 Dify Docker 部署方式的需要注意网络配置，确保 Dify 容器可以访问到 Xinference 的端点，Dify 容器内部无法访问到 localhost，需要使用宿主机 IP 地址。By making these changes, you ensure that the voice parameter is included in the XinferenceModelExtraParameter class and is correctly extracted from the response and passed through the relevant methods. Additionally, ensure that your network settings allow Dify to access the Xinference endpoint [1][2][3].
To continue talking to Dosu, mention @dosu.
ic-xu
cycwll
hunterhome
crazywoola
Self Checks
Dify version
0.6.16
Cloud or Self Hosted
Self Hosted (Docker)
Steps to reproduce
I used the cosyvoice model via xinference, dify use it as text to speech model. No matter what I configure the voice for Text to Speech, the value of the voice passed from dify to xinference is always empty
{'intput': '记得联系技术支持寻求进一步的帮助。', 'voice': '', 'response_format': 'mp3', 'speed': 1.0, 'stream': False}
✔️ Expected Behavior
No response
❌ Actual Behavior
No response