Qwen1.5-7B-Chat 使用API呼叫 completions 功能未能生成下文

提交前必须检查以下项目 | The following items must be checked before submission

[X] 请确保使用的是仓库最新代码（git pull），一些问题已被解决和修复。 | Make sure you are using the latest code from the repository (git pull), some issues have already been addressed and fixed.
[X] 我已阅读项目文档和FAQ章节并且已在Issue中对问题进行了搜索，没有找到相似问题和解决方案 | I have searched the existing issues / discussions

问题类型 | Type of problem

模型推理和部署 | Model inference and deployment

操作系统 | Operating system

Linux

详细描述问题 | Detailed description of the problem

Qwen1.5-7B-Chat模型，使用API 向 /completions 接口 post 以下信息

{
    "model": "qwen2",
    "prompt": "How are you?",
    "max_tokens": 1024,
    "temperature": 0
}

返回的结果中未生成下文内容，text为空：

{
    "id": "cmpl-3bb4aed3-fc65-4385-82b5-e559d03a7fa9",
    "choices": [
        {
            "finish_reason": "stop",
            "index": 0,
            "logprobs": null,
            "text": ""
        }
    ],
    "created": 1708351606,
    "model": "qwen2",
    "object": "text_completion",
    "system_fingerprint": null,
    "usage": {
        "completion_tokens": 0,
        "prompt_tokens": 4,
        "total_tokens": 4
    }
}

Dependencies

python=3.8.18

openai==1.12.0 pydantic==1.10.13 fastapi==0.104.1 python-dotenv==1.0.0 transformers==4.37.2 peft==0.8.2 accelerate==0.21.0 sentence-transformers==2.2.2 transformers_stream_generator==0.0.4 einops==0.6.1 sse-starlette==1.6.1 starlette-context==0.3.6 torch==2.0.1

运行日志或截图 | Runtime logs or screenshots

No response

是的，上面这个例子，调整temperature 的情况下有概率会有回复。

还有一个是我测试Langchain的APIChain的例子，也存在这样的情况。

import os
from langchain.chains import APIChain
from langchain.chains.api import open_meteo_docs
from langchain_openai import OpenAI

os.environ['OPENAI_API_BASE'] = 'http://127.0.0.1:8000/v1'
os.environ['OPENAI_API_KEY'] = 'OPENAI_API_KEY'

llm = OpenAI(model='qwen2',temperature=0)
chain = APIChain.from_llm_and_api_docs(
    llm,
    open_meteo_docs.OPEN_METEO_DOCS,
    verbose=True,
    limit_to_domains=["https://api.open-meteo.com/"],
)
chain.run(
    "What is the weather like right now in Munich, Germany in degrees Fahrenheit?"
)

生成的API呼叫的示例是向/completions post以下的内容，这个的返回结果 text 也是没有内容的。

{
    "model": "qwen2",
    "prompt": "You are given the below API Documentation:\nBASE URL: https://api.open-meteo.com/\n\nAPI Documentation\nThe API endpoint /v1/forecast accepts a geographical coordinate, a list of weather variables and responds with a JSON hourly weather forecast for 7 days. Time always starts at 0:00 today and contains 168 hours. All URL parameters are listed below:\n\nParameter\tFormat\tRequired\tDefault\tDescription\nlatitude, longitude\tFloating point\tYes\t\tGeographical WGS84 coordinate of the location\nhourly\tString array\tNo\t\tA list of weather variables which should be returned. Values can be comma separated, or multiple &hourly= parameter in the URL can be used.\ndaily\tString array\tNo\t\tA list of daily weather variable aggregations which should be returned. Values can be comma separated, or multiple &daily= parameter in the URL can be used. If daily weather variables are specified, parameter timezone is required.\ncurrent_weather\tBool\tNo\tfalse\tInclude current weather conditions in the JSON output.\ntemperature_unit\tString\tNo\tcelsius\tIf fahrenheit is set, all temperature values are converted to Fahrenheit.\nwindspeed_unit\tString\tNo\tkmh\tOther wind speed speed units: ms, mph and kn\nprecipitation_unit\tString\tNo\tmm\tOther precipitation amount units: inch\ntimeformat\tString\tNo\tiso8601\tIf format unixtime is selected, all time values are returned in UNIX epoch time in seconds. Please note that all timestamp are in GMT+0! For daily values with unix timestamps, please apply utc_offset_seconds again to get the correct date.\ntimezone\tString\tNo\tGMT\tIf timezone is set, all timestamps are returned as local-time and data is returned starting at 00:00 local-time. Any time zone name from the time zone database is supported. If auto is set as a time zone, the coordinates will be automatically resolved to the local time zone.\npast_days\tInteger (0-2)\tNo\t0\tIf past_days is set, yesterday or the day before yesterday data are also returned.\nstart_date\nend_date\tString (yyyy-mm-dd)\tNo\t\tThe time interval to get weather data. A day must be specified as an ISO8601 date (e.g. 2022-06-30).\nmodels\tString array\tNo\tauto\tManually select one or more weather models. Per default, the best suitable weather models will be combined.\n\nHourly Parameter Definition\nThe parameter &hourly= accepts the following values. Most weather variables are given as an instantaneous value for the indicated hour. Some variables like precipitation are calculated from the preceding hour as an average or sum.\n\nVariable\tValid time\tUnit\tDescription\ntemperature_2m\tInstant\t°C (°F)\tAir temperature at 2 meters above ground\nsnowfall\tPreceding hour sum\tcm (inch)\tSnowfall amount of the preceding hour in centimeters. For the water equivalent in millimeter, divide by 7. E.g. 7 cm snow = 10 mm precipitation water equivalent\nrain\tPreceding hour sum\tmm (inch)\tRain from large scale weather systems of the preceding hour in millimeter\nshowers\tPreceding hour sum\tmm (inch)\tShowers from convective precipitation in millimeters from the preceding hour\nweathercode\tInstant\tWMO code\tWeather condition as a numeric code. Follow WMO weather interpretation codes. See table below for details.\nsnow_depth\tInstant\tmeters\tSnow depth on the ground\nfreezinglevel_height\tInstant\tmeters\tAltitude above sea level of the 0°C level\nvisibility\tInstant\tmeters\tViewing distance in meters. Influenced by low clouds, humidity and aerosols. Maximum visibility is approximately 24 km.\nUsing this documentation, generate the full API url to call for answering the user question.\nYou should build the API url in order to get a response that is as short as possible, while still getting the necessary information to answer the question. Pay attention to deliberately exclude any unnecessary pieces of data in the API call.\n\nQuestion:What is the weather like right now in Munich, Germany in degrees Fahrenheit?\nAPI url:",
    "max_tokens": 4096,
    "temperature": 0
}

返回结果中text为空。

{
    "id": "cmpl-18126258-f48c-4022-ab27-6b98fa940c35",
    "choices": [
        {
            "finish_reason": "stop",
            "index": 0,
            "logprobs": null,
            "text": ""
        }
    ],
    "created": 1708395634,
    "model": "qwen2",
    "object": "text_completion",
    "system_fingerprint": null,
    "usage": {
        "completion_tokens": 0,
        "prompt_tokens": 867,
        "total_tokens": 867
    }
}

xusenlinzy / api-for-open-llm