search

langgenius / dify

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.
https://dify.ai
Other
50.28k stars 7.2k forks source link

"[xinference] Server Unavailable Error, Internal Server Error" in workflow #3315

Closed liuzhenghua closed 6 months ago

liuzhenghua commented 6 months ago

Self Checks

Dify version

0.6.1

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

  1. create a workflow from template "三步翻译工作流"
  2. change the llm provider to qwen1.5-chat which provided by xinference.
  3. click the run button

the model is ok in chat bot. ![Uploading image.png…]()

✔️ Expected Behavior

execute success.

❌ Actual Behavior

image

liuzhenghua commented 6 months ago

image

dosubot[bot] commented 6 months ago

你好，@liuzhenghua！很高兴再次见到你。

The error message you're encountering, "Table 'student' references invalid table(s) or column(s) or function(s) or definer/invoker of view lack rights to use them," typically indicates a problem with a database operation. This could be due to a few reasons:

  1. Non-existent Table/Column/Function: The 'student' table, or a column/function it relies on, might not exist in your database schema. Ensure all referenced database objects are present and correctly named.

  2. Insufficient Permissions: The database user might not have the necessary permissions to access the 'student' table or related objects. Adjust the permissions by granting the required access to your database user.

  3. Database Configuration Issue: There might be a misconfiguration in your application's database connection settings, leading it to connect to the wrong database or schema where the 'student' table does not exist.

To address these issues, verify the existence and correct naming of the 'student' table and any related objects in your database schema. Also, ensure the database user has the appropriate permissions and that your application's database connection settings are correctly configured.

Regarding the chatbot interface, without specific error messages or a detailed description of unexpected behavior, it's challenging to diagnose potential issues. If there's a particular problem you're facing with the chatbot interface, please provide more details for a more targeted solution.

For further assistance, consider reviewing the following resources in the Dify repository:

These resources might offer additional insights into configuring your workflow and resolving the encountered issues.

Sources

#### About Dosu This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.
liuzhenghua commented 6 months ago

Internal Server Error Traceback (most recent call last): File "/app/api/core/model_runtime/model_providers/__base/large_language_model.py", line 96, in invoke result = self._invoke(model, credentials, prompt_messages, model_parameters, tools, stop, stream, user) File "/app/api/core/model_runtime/model_providers/xinference/llm/llm.py", line 79, in _invoke return self._generate( File "/app/api/core/model_runtime/model_providers/xinference/llm/llm.py", line 413, in _generate resp = client.chat.completions.create( File "/usr/local/lib/python3.10/site-packages/openai/_utils/_utils.py", line 275, in wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/openai/resources/chat/completions.py", line 667, in create return self._post( File "/usr/local/lib/python3.10/site-packages/openai/_base_client.py", line 1208, in post return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)) File "/usr/local/lib/python3.10/site-packages/openai/_base_client.py", line 897, in request return self._request( File "/usr/local/lib/python3.10/site-packages/openai/_base_client.py", line 973, in _request return self._retry_request( File "/usr/local/lib/python3.10/site-packages/openai/_base_client.py", line 1021, in _retry_request return self._request( File "/usr/local/lib/python3.10/site-packages/openai/_base_client.py", line 973, in _request return self._retry_request( File "/usr/local/lib/python3.10/site-packages/openai/_base_client.py", line 1021, in _retry_request return self._request( File "/usr/local/lib/python3.10/site-packages/openai/_base_client.py", line 973, in _request return self._retry_request( File "/usr/local/lib/python3.10/site-packages/openai/_base_client.py", line 1021, in _retry_request return self._request( File "/usr/local/lib/python3.10/site-packages/openai/_base_client.py", line 988, in _request raise self._make_status_error_from_response(err.response) from None

liuzhenghua commented 6 months ago

but no logs in xinference

andylzming commented 6 months ago

The same problem。

112

DEBUG:httpcore.http11:receive_response_headers.complete return_value=(b'HTTP/1.1', 500, b'Internal Server Error', [(b'date', b'Thu, 11 Apr 2024 20:12:17 GMT'), (b'server', b'uvicorn'), (b'content-length', b'21'), (b'content-type', b'text/plain; charset=utf-8')])
INFO:httpx:HTTP Request: POST http://127.0.0.1:59997/v1/chat/completions "HTTP/1.1 500 Internal Server Error"
DEBUG:openai._base_client:HTTP Request: POST http://127.0.0.1:59997/v1/chat/completions "500 Internal Server Error"
DEBUG:openai._base_client:Encountered httpx.HTTPStatusError
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/openai/_base_client.py", line 967, in _request
    response.raise_for_status()
  File "/usr/local/lib/python3.10/site-packages/httpx/_models.py", line 749, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Server error '500 Internal Server Error' for url 'http://127.0.0.1:59997/v1/chat/completions'
For more information check: https://httpstatuses.com/500

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/openai/_base_client.py", line 967, in _request
    response.raise_for_status()
  File "/usr/local/lib/python3.10/site-packages/httpx/_models.py", line 749, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Server error '500 Internal Server Error' for url 'http://127.0.0.1:59997/v1/chat/completions'
For more information check: https://httpstatuses.com/500

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/openai/_base_client.py", line 967, in _request
    response.raise_for_status()
  File "/usr/local/lib/python3.10/site-packages/httpx/_models.py", line 749, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Server error '500 Internal Server Error' for url 'http://127.0.0.1:59997/v1/chat/completions'
For more information check: https://httpstatuses.com/500

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/openai/_base_client.py", line 967, in _request
    response.raise_for_status()
  File "/usr/local/lib/python3.10/site-packages/httpx/_models.py", line 749, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Server error '500 Internal Server Error' for url 'http://127.0.0.1:59997/v1/chat/completions'
For more information check: https://httpstatuses.com/500
DEBUG:httpcore.http11:receive_response_body.started request=<Request [b'POST']>
DEBUG:httpcore.http11:receive_response_body.complete
DEBUG:httpcore.http11:response_closed.started
DEBUG:httpcore.http11:response_closed.complete
DEBUG:httpcore.connection:close.started
DEBUG:httpcore.connection:close.complete
DEBUG:openai._base_client:Re-raising status error
DEBUG:core.app.task_pipeline.based_generate_task_pipeline:error: Run failed: Node LLM run failed: [xinference] Server Unavailable Error, Internal Server Error

INFO 04-11 16:12:11 async_llm_engine.py:379] Received request c7c6bb42-f83f-11ee-8eb7-80615f20f615: prompt: '<|im_start|>system\n\n    ### Job Description\',\n    You are a text classification engine that analyzes text data and assigns categories based on user input or automatically determined categories.\n    ### Task\n    Your task is to assign one categories ONLY to the input text and only one category may be assigned returned in the output.Additionally, you need to extract the key words from the text that are related to the classification.\n    ### Format\n    The input text is in the variable text_field.Categories are specified as a comma-separated list in the variable categories or left empty for automatic determination.Classification instructions may be included to improve the classification accuracy.\n    ### Constraint\n    DO NOT include anything other than the JSON array in your response.\n    ### Memory\n    Here is the chat histories between human and assistant, inside <histories></histories> XML tags.\n    <histories>\n    \n    </histories>\n<|im_end|>\n<|im_start|>user\n    { "input_text": ["I recently had a great experience with your company. The service was prompt and the staff was very friendly."],\n    "categories": ["Customer Service, Satisfaction, Sales, Product"],\n    "classification_instructions": ["classify the text based on the feedback provided by customer"]}```JSON<|im_end|>\n<|im_start|>assistant\n    {"keywords": ["recently", "great experience", "company", "service", "prompt", "staff", "friendly"],\n    "categories": ["Customer Service"]}```<|im_end|>\n<|im_start|>user\n    {"input_text": ["bad service, slow to bring the food"],\n    "categories": ["Food Quality, Experience, Price" ], \n    "classification_instructions": []}```JSON<|im_end|>\n<|im_start|>assistant\n    {"keywords": ["bad service", "slow", "food", "tip", "terrible", "waitresses"],\n    "categories": ["Experience""]}```<|im_end|>\n<|im_start|>user\n    \'{"input_text": ["电量计算"],\',\n    \'"categories": ["电量计算,其他" ], \',\n    \'"classification_instructions": [""]}```JSON\'<|im_end|>\n<|im_start|>assistant\n', sampling params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.5, top_p=1.0, top_k=-1, min_p=0.0, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=['<|endoftext|>', '<|im_start|>', '<|im_end|>'], stop_token_ids=[151643, 151644, 151645], include_stop_str_in_output=False, ignore_eos=False, max_tokens=6801, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True), prompt token ids: None.
INFO 04-11 16:12:11 llm_engine.py:653] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.7%, CPU KV cache usage: 0.0%
INFO 04-11 16:12:11 async_llm_engine.py:111] Finished request c7c6bb42-f83f-11ee-8eb7-80615f20f615.
hunterhome commented 6 months ago

Traceback (most recent call last): File "/home/rjsoft/anaconda3/envs/xinference/lib/python3.9/site-packages/uvicorn/protocols/http/httptools_impl.py", line 426, in run_asgi result = await app( # type: ignore[func-returns-value] File "/home/rjsoft/anaconda3/envs/xinference/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in call return await self.app(scope, receive, send) File "/home/rjsoft/anaconda3/envs/xinference/lib/python3.9/site-packages/fastapi/applications.py", line 1106, in call await super().call(scope, receive, send) File "/home/rjsoft/anaconda3/envs/xinference/lib/python3.9/site-packages/starlette/applications.py", line 122, in call await self.middleware_stack(scope, receive, send) File "/home/rjsoft/anaconda3/envs/xinference/lib/python3.9/site-packages/starlette/middleware/errors.py", line 184, in call raise exc File "/home/rjsoft/anaconda3/envs/xinference/lib/python3.9/site-packages/starlette/middleware/errors.py", line 162, in call await self.app(scope, receive, _send) File "/home/rjsoft/anaconda3/envs/xinference/lib/python3.9/site-packages/aioprometheus/asgi/middleware.py", line 184, in call await self.asgi_callable(scope, receive, wrapped_send) File "/home/rjsoft/anaconda3/envs/xinference/lib/python3.9/site-packages/starlette/middleware/cors.py", line 83, in call await self.app(scope, receive, send) File "/home/rjsoft/anaconda3/envs/xinference/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 79, in call raise exc File "/home/rjsoft/anaconda3/envs/xinference/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 68, in call await self.app(scope, receive, sender) File "/home/rjsoft/anaconda3/envs/xinference/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in call raise e File "/home/rjsoft/anaconda3/envs/xinference/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in call await self.app(scope, receive, send) File "/home/rjsoft/anaconda3/envs/xinference/lib/python3.9/site-packages/starlette/routing.py", line 718, in call await route.handle(scope, receive, send) File "/home/rjsoft/anaconda3/envs/xinference/lib/python3.9/site-packages/starlette/routing.py", line 276, in handle await self.app(scope, receive, send) File "/home/rjsoft/anaconda3/envs/xinference/lib/python3.9/site-packages/starlette/routing.py", line 66, in app response = await func(request) File "/home/rjsoft/anaconda3/envs/xinference/lib/python3.9/site-packages/fastapi/routing.py", line 274, in app raw_response = await run_endpoint_function( File "/home/rjsoft/anaconda3/envs/xinference/lib/python3.9/site-packages/fastapi/routing.py", line 191, in run_endpoint_function return await dependant.call(**values) File "/home/rjsoft/anaconda3/envs/xinference/lib/python3.9/site-packages/xinference/api/restful_api.py", line 1285, in create_chat_completion assert non_system_messages AssertionError

liuzhenghua commented 6 months ago

while no 'user' role input, xinference returns 500 image

liuzhenghua commented 6 months ago

image

liuzhenghua commented 6 months ago

If I add USER msg in wokflow node, it works. image image

liuzhenghua commented 6 months ago

It would be fixed by xinference

JokerQyou commented 3 months ago

Just figured out myself, at least a single message of user role is required to call a chat model (tested with Qwen series). If using text generation model this would not be necessary.