search

PaddlePaddle / PaddleNLP

👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.
https://paddlenlp.readthedocs.io
Apache License 2.0
11.71k stars 2.86k forks source link

[Docs]: 示例里 8010 和 8011 写反了 #8646

Open Skypow2012 opened 1 week ago

Skypow2012 commented 1 week ago

软件环境

- paddlepaddle:
- paddlepaddle-gpu: 
- paddlenlp:

详细描述

llm/README.mnd
python -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" ./predict/flask_server.py \
    --model_name_or_path meta-llama/Llama-2-7b-chat \
    --port 8010 \
    --flask_port 8011 \
    --dtype "float16"
这里建议 port 8011, flask_port 8010，不然按这个 request_flask_server.py 直接跑会报错容易误解