search

IEIT-Yuan / Yuan-2.0

Yuan 2.0 Large Language Model
Other
681 stars 85 forks source link

Yuan2.0 推理 API 部署 #122

Closed Emirateyang closed 7 months ago

Emirateyang commented 7 months ago

环境： Ubuntu 22.04 GPU

参考 https://github.com/IEIT-Yuan/Yuan-2.0/blob/main/docs/inference_server_cn.md#HuggingFace%E6%A8%A1%E5%9E%8B%E6%8E%A8%E7%90%86API%E9%83%A8%E7%BD%B2 进行推理API的部署。 启动后，使用Curl进行测试时 curl http://127.0.0.1:8000/yuan -X PUT \ --header 'Content-Type: application/json' \ --data '{"ques_list":[{"id":"000","ques":"请帮忙作一首诗，主题是冬至"}], "tokens_to_generate":500, "top_k":5}'

返回如下错误: {“errCode”:2, "excetionMsg": "LlamaRotaryEmbedding.forward() missing 1 required positional argument:'position_ids'", "flag":false}

我还需要注意点什么？怎么解决这个问题

Emirateyang commented 7 months ago

same as #123 current version not match the transformers version 4.38 or later, downgrade to 4.37.2 will fix this issue