ishelaputov
commented
3 months ago Hello! The problem is some unescaped text characters that are passed to the input.
It helped to force the text to be converted to a json string, for example:
output_text = json.dumps(result["output_text"], ensure_ascii=False)Example request:
curl --location 'http://10.146.240.74:30000/v1/chat/completions' \
--header 'Content-Type: application/json' \
--data '{
"messages": [
{
"content": "You are an assistant who can write the user'\''s last response to a file.\nDetermine the class name from the user description and use it as the name of the txt file, for example CreateIssues.txt.\nSave the file in the raw_data folder.\nRecord the content unchanged as provided by the user and nothing else.\nReturn only the path to the file, for example /raw_data/CreateIssues.txt. Work autonomously according to your specialty, using the tools available to you. Answer briefly and only in your specialty.",
"role": "system"
},
{
"role": "user",
"content": "Analyze the content and write to file"
},
{
"role": "user",
"name": "controller_analizer",
"content": "Описание контроллера '\''CreateIssuesController'\''\\nМетоды:\\nGET /api/jira/issues/createFromExcel\\nНазначение метода: Метод массового создания задач в Jira из Excel файла.\\nЗаголовки запроса:\\nContent-Type: multipart/form-data\\nВходные параметры:\\nПараметр: file\\n- Описание: xlsx файл с задачами, которые надо создать\\n- Тип: MultipartFile\\n- Обязательность: Да\\n- Пример значение: файл.xlsx\\nПример запроса:\\nPOST /api/jira/issues/createFromExcel HTTP/1.1\\nHost: example.com\\nContent-Type: multipart/form-data; boundary=---------------------------1234567890\\n-----------------------------1234567890\\nContent-Disposition: form-data; name=\"file\"; filename=\"file.xlsx\"\\nContent-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet\\n... файл.xlsx...\\n-----------------------------1234567890--\\nВыходные параметры:\\nПараметр: response\\n- Описание: Список успешно созданных задач и список не созданных задач с описанием ошибок\\n- Тип: JiraTaskCreateResponse\\n- Обязательность: Да\\n- Пример значение: {\\n\\t\"createdTasks\": [...],\\n\\t\"errors\": [...]}\\nПример ответа:\\nHTTP/1.1 201 Created\\nContent-Type: application/json\\n{\\n\\t\"createdTasks\": [...],\\n\\t\"errors\": [...]}\\nКоды ответа:\\n201 Created - успешное создание задач\\n400 Bad Request - ошибка при создании задач"
}
],
"model": "/meta-llama/Meta-Llama-3-8B-Instruct",
"max_tokens": 1024,
"temperature": 0.01,
"n": 50,
"top_p": 0.9,
"stream": false,
"tools": [
{
"type": "function",
"function": {
"name": "write_document",
"description": "Create and save a text document. Return path of the saved document file.",
"parameters": {
"type": "object",
"properties": {
"content": {
"description": "Text content to be written into the document.",
"type": "string"
},
"file_name": {
"description": "File path to save the document.",
"type": "string"
}
},
"required": [
"content",
"file_name"
]
}
}
}
],
"tool_choice": "auto"
}'Response:
{
"id": "",
"object": "chat.completion",
"created": 1719749467,
"model": "/meta-llama/Meta-Llama-3-8B-Instruct",
"system_fingerprint": "2.1.0-sha-192d49a",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"tool_calls": [
{
"id": "0",
"type": "function",
"function": {
"description": null,
"name": "write_document",
"arguments": {
"content": "Описание контроллера 'CreateIssuesController'\nМетоды:\nGET /api/jira/issues/createFromExcel\nНазначение метода: Метод массового создания задач в Jira из Excel файла.\nЗаголовки запроса:\nContent-Type: multipart/form-data\nВходные параметры:\nПараметр: file\n- Описание: xlsx файл с задачами, которые надо создать\n- Тип: MultipartFile\n- Обязательность: Да\n- Пример значение: файл.xlsx\nПример запроса:\nPOST /api/jira/issues/createFromExcel HTTP/1.1\nHost: example.com\nContent-Type: multipart/form-data; boundary=---------------------------1234567890\n-----------------------------1234567890\nContent-Disposition: form-data; name=\"file\"; filename=\"file.xlsx\"\nContent-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet\n... файл.xlsx...\n-----------------------------1234567890--\nВыходные параметры:\nПараметр: response\n- Описание: Список успешно созданных задач и список не созданных задач с описанием ошибок\n- Тип: JiraTaskCreateResponse\n- Обязательность: Да\n- Пример значение: {\n\t\"createdTasks\": [...],\n\t\"errors\": [...]}\nПример ответа:\nHTTP/1.1 201 Created\nContent-Type: application/json\n{\n\t\"createdTasks\": [...],\n\t\"errors\": [...]}\nКоды ответа:\n201 Created - успешное создание задач\n400 Bad Request - ошибка при создании задач",
"file_name": "/raw_data/CreateIssues.txt"
}
}
}
]
},
"logprobs": null,
"finish_reason": "eos_token"
}
],
"usage": {
"prompt_tokens": 695,
"completion_tokens": 380,
"total_tokens": 1075
}
}
RonanKMcGovern
System Info
Hello! Thank you very much for your product, very helpful!
System Info:
Model info:
TGI Version: 2.1.0
Information
Tasks
Reproduction
When I execute the following query with the need to call a tool by the model:
I get the error:
If you call the same request with
"stream": true, then this is the result: output_raw.txt output.txtIn the file output.txt all the values of
argumentsare collected in one line and here’s what’s strange: 1) the JSON Schema of my and, as I understand it, default tool is added to the text for thecontentparameter my tool below 2) JSON Schema does not have the last closing character}Expected behavior
Expected:
Thanks!