TODAM-tw / todam-ticket-system

The frontend with gradio and combined with the API endpoints for the ticket system.
Apache License 2.0
0 stars 0 forks source link

[Enhancement] Handling Errors Caused by Model Output Formatting #28

Closed 1chooo closed 1 month ago

1chooo commented 1 month ago

現有處理 content["text"] 的作法可能會因為 Model 的 output 產生 error 因為 Model 可能會產生 ```json ```content["text"] 裡的內容包裝起來,因此可以加上 if-elif 判斷去特別處理這個例外以避免產生 Error 導致系統無法正常運行。

[!NOTE] 或許還可以同時加上 ``` ``` 的例外處理,以更全面地避免模型加入 ```

https://github.com/TODAM-tw/todam-ticket-system/blob/2862c80f4fb6d63908c9c203d15e61a7a769ce1b/app/cases/summarized_content.py#L69-L72

可參考現有文件作法:

https://github.com/TODAM-tw/todam-ticket-system/blob/2862c80f4fb6d63908c9c203d15e61a7a769ce1b/docs/bedrock_model.md?plain=1#L214-L242

1chooo commented 1 month ago

最近 prod stage 的 Model Output 格式高頻率出錯,會導致呼叫 get_summarized_ticket_content 時很容易拿到 Error

Traceback (most recent call last):
  File "/Users/hugolin/Developer/todam-ticket-system/venv/lib/python3.11/site-packages/gradio/queueing.py", line 527, in process_events
    response = await route_utils.call_process_api(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hugolin/Developer/todam-ticket-system/venv/lib/python3.11/site-packages/gradio/route_utils.py", line 270, in call_process_api
    output = await app.get_blocks().process_api(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hugolin/Developer/todam-ticket-system/venv/lib/python3.11/site-packages/gradio/blocks.py", line 1897, in process_api
    data = await self.postprocess_data(fn_index, result["prediction"], state)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hugolin/Developer/todam-ticket-system/venv/lib/python3.11/site-packages/gradio/blocks.py", line 1673, in postprocess_data
    self.validate_outputs(fn_index, predictions)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hugolin/Developer/todam-ticket-system/venv/lib/python3.11/site-packages/gradio/blocks.py", line 1649, in validate_outputs
    raise ValueError(
ValueError: An event handler (get_summarized_ticket_content) didn't receive enough output values (needed: 2, received: 1).
Wanted outputs:
    [<gradio.components.html.HTML object at 0x115e82f10>, <gradio.components.textbox.Textbox object at 0x115e2c4d0>]
Received outputs:
    ["Error: Something went wrong with the API"]

Related Issue: #17