search

tensorchord / modelz-llm

OpenAI compatible API for LLMs and embeddings (LLaMA, Vicuna, ChatGLM and many others)
https://modelz.ai
Apache License 2.0
264 stars 26 forks source link

bug: Failed to generate outputs #97

Open gaocegege opened 1 year ago

gaocegege commented 1 year ago

Get assertion error with https://github.com/kharvd/gpt-cli

Traceback (most recent call last):
  File "/home/gaocegege/applications/miniconda3/envs/dev/bin/gpt", line 8, in <module>
    sys.exit(main())
  File "/home/gaocegege/applications/miniconda3/envs/dev/lib/python3.9/site-packages/gptcli/gpt.py", line 189, in main
    run_interactive(args, assistant)
  File "/home/gaocegege/applications/miniconda3/envs/dev/lib/python3.9/site-packages/gptcli/gpt.py", line 237, in run_interactive
    session.loop(input_provider)
  File "/home/gaocegege/applications/miniconda3/envs/dev/lib/python3.9/site-packages/gptcli/session.py", line 183, in loop
    while self.process_input(*input_provider.get_user_input()):
  File "/home/gaocegege/applications/miniconda3/envs/dev/lib/python3.9/site-packages/gptcli/session.py", line 175, in process_input
    response_saved = self._respond(args)
  File "/home/gaocegege/applications/miniconda3/envs/dev/lib/python3.9/site-packages/gptcli/session.py", line 109, in _respond
    for response in completion_iter:
  File "/home/gaocegege/applications/miniconda3/envs/dev/lib/python3.9/site-packages/gptcli/openai.py", line 20, in complete
    openai.ChatCompletion.create(
  File "/home/gaocegege/applications/miniconda3/envs/dev/lib/python3.9/site-packages/openai/api_resources/chat_completion.py", line 25, in create
    return super().create(*args, **kwargs)
  File "/home/gaocegege/applications/miniconda3/envs/dev/lib/python3.9/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 165, in create
    assert not isinstance(response, OpenAIResponse)
AssertionError
An uncaught exception occurred. Please report this issue on GitHub.
Traceback (most recent call last):
  File "/home/gaocegege/applications/miniconda3/envs/dev/bin/gpt", line 8, in <module>
    sys.exit(main())
  File "/home/gaocegege/applications/miniconda3/envs/dev/lib/python3.9/site-packages/gptcli/gpt.py", line 189, in main
    run_interactive(args, assistant)
  File "/home/gaocegege/applications/miniconda3/envs/dev/lib/python3.9/site-packages/gptcli/gpt.py", line 237, in run_interactive
    session.loop(input_provider)
  File "/home/gaocegege/applications/miniconda3/envs/dev/lib/python3.9/site-packages/gptcli/session.py", line 183, in loop
    while self.process_input(*input_provider.get_user_input()):
  File "/home/gaocegege/applications/miniconda3/envs/dev/lib/python3.9/site-packages/gptcli/session.py", line 175, in process_input
    response_saved = self._respond(args)
  File "/home/gaocegege/applications/miniconda3/envs/dev/lib/python3.9/site-packages/gptcli/session.py", line 109, in _respond
    for response in completion_iter:
  File "/home/gaocegege/applications/miniconda3/envs/dev/lib/python3.9/site-packages/gptcli/openai.py", line 20, in complete
    openai.ChatCompletion.create(
  File "/home/gaocegege/applications/miniconda3/envs/dev/lib/python3.9/site-packages/openai/api_resources/chat_completion.py", line 25, in create
    return super().create(*args, **kwargs)
  File "/home/gaocegege/applications/miniconda3/envs/dev/lib/python3.9/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 165, in create
    assert not isinstance(response, OpenAIResponse)
AssertionError
gaocegege commented 1 year ago

The server actually generated the output: 

2023-08-02T16:31:53.058244518+08:00 stderr F 2023-08-02 08:31:53,058 - 1 - INFO - falcon_service.py:46 - ChatResponse(id='2bfbb49d-3d5b-4333-adfe-93531fd70b5f', object='chat', created=datetime.datetime(2023, 8, 2, 8, 31, 52, 982729), model='bigscience/bloomz-560m', usage=TokenUsage(prompt_tokens=11, completion_tokens=8, total_tokens=19), choices=[ChatChoice(message=ChatMessage(content='ive been working for you for a while', role=<Role.ASSISTANT: 'assistant'>, name=''), index=0, finish_reason='stop')])
gaocegege commented 1 year ago

/assign @kemingy