intel-analytics / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Baichuan, Mixtral, Gemma, Phi, MiniCPM, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, GraphRAG, DeepSpeed, vLLM, FastChat, Axolotl, etc.
Apache License 2.0
6.61k stars 1.26k forks source link

Qwen-7B-Chat answer quality #9563

Open KiwiHana opened 10 months ago

KiwiHana commented 10 months ago

The answer quality of Qwen-7B-Chat get some wrong on MTL iGPU 问: 校长说:“校服上除了校徽别别别的” 这句话里校长表达什么意思? 答: 't know

Package                       Version
----------------------------- ------------------
absl-py                       2.0.0
accelerate                    0.23.0
aiofiles                      23.2.1
aiohttp                       3.8.6
aiosignal                     1.3.1
altair                        5.1.2
annotated-types               0.6.0
antlr4-python3-runtime        4.9.3
anyio                         3.7.1
astor                         0.8.1
asttokens                     2.4.0
async-timeout                 4.0.3
attrs                         23.1.0
audioread                     3.0.1
Babel                         2.13.0
backcall                      0.2.0
bce-python-sdk                0.8.92
bigdl-core-xe                 2.4.0.dev0
bigdl-llm                     2.4.0b20231029
blinker                       1.6.3
bmm-xe                        0.0.0
bokeh                         3.3.0
boltons                       23.0.0
Bottleneck                    1.3.7
braceexpand                   0.1.7
certifi                       2023.7.22
cffi                          1.16.0
charset-normalizer            3.3.2
click                         8.1.7
colorama                      0.4.6
coloredlogs                   15.0.1
colorlog                      6.7.0
contourpy                     1.1.1
cycler                        0.12.1
Cython                        3.0.4
datasets                      2.14.5
decorator                     5.1.1
diffusers                     0.21.2
dill                          0.3.4
Distance                      0.1.3
editdistance                  0.6.2
einops                        0.7.0
exceptiongroup                1.1.3
executing                     2.0.0
fastapi                       0.104.0
ffmpy                         0.3.1
filelock                      3.13.1
Flask                         3.0.0
Flask-Babel                   2.0.0
Flask-Cors                    4.0.0
flatbuffers                   23.5.26
fonttools                     4.43.1
frozenlist                    1.4.0
fsspec                        2023.6.0
ftfy                          6.1.1
future                        0.18.3
g2p-en                        2.1.0
g2pM                          0.1.2.5
gradio                        3.41.1
gradio_client                 0.5.0
h11                           0.14.0
h5py                          3.10.0
httpcore                      0.18.0
httpx                         0.25.0
huggingface-hub               0.18.0
humanfriendly                 10.0
HyperPyYAML                   1.2.2
idna                          3.4
importlib-metadata            6.8.0
importlib-resources           6.1.0
inflect                       7.0.0
intel-extension-for-pytorch   2.0.110+gitba7f6c1
intervaltree                  3.1.0
ipython                       8.16.1
itsdangerous                  2.1.2
jedi                          0.19.1
jieba                         0.42.1
Jinja2                        3.1.2
joblib                        1.3.2
jsonlines                     4.0.0
jsonschema                    4.19.1
jsonschema-specifications     2023.7.1
kaldiio                       2.18.0
kiwisolver                    1.4.5
langid                        1.1.6
latex2mathml                  3.76.0
librosa                       0.8.1
llvmlite                      0.41.1
loguru                        0.7.2
lxml                          4.9.3
Markdown                      3.5
markdown-it-py                3.0.0
MarkupSafe                    2.1.3
matplotlib                    3.8.0
matplotlib-inline             0.1.6
mdtex2html                    1.2.0
mdurl                         0.1.2
mido                          1.3.0
mock                          5.1.0
more-itertools                10.1.0
mpmath                        1.3.0
multidict                     6.0.4
multiprocess                  0.70.12.2
nara-wpe                      0.0.9
networkx                      3.2.1
nltk                          3.8.1
note-seq                      0.0.3
numba                         0.58.1
numpy                         1.23.0
omegaconf                     2.3.0
onnx                          1.14.1
onnxruntime                   1.11.0
openai-whisper                20230918
OpenCC                        1.1.7
opencc-python-reimplemented   0.1.7
opencv-python                 4.8.1.78
openvino                      2023.1.0
openvino-telemetry            2023.2.1
opt-einsum                    3.3.0
optimum                       1.13.2
optimum-intel                 1.11.0
orjson                        3.9.9
packaging                     23.2
paddle-bfloat                 0.1.7
paddle2onnx                   1.0.6
paddleaudio                   1.1.0
paddlefsl                     1.1.0
paddlenlp                     2.5.2
paddlepaddle                  2.4.0
paddlesde                     0.2.5
paddleslim                    2.3.4
paddlespeech                  1.3.0
paddlespeech-feat             0.1.0
pandas                        2.1.1
parameterized                 0.9.0
parso                         0.8.3
pathos                        0.2.8
pattern-singleton             1.2.0
pickleshare                   0.7.5
Pillow                        10.1.0
pip                           23.3
platformdirs                  3.11.0
pooch                         1.7.0
portalocker                   2.8.2
pox                           0.3.3
ppdiffusers                   0.19.3
ppft                          1.7.6.7
praatio                       5.0.0
pretty-midi                   0.2.10
prettytable                   3.9.0
prompt-toolkit                3.0.39
protobuf                      3.20.0
psutil                        5.9.6
pure-eval                     0.2.2
py-cpuinfo                    9.0.0
pyarrow                       13.0.0
PyAudio                       0.2.13
pybind11                      2.11.1
pycparser                     2.21
pycryptodome                  3.19.0
pydantic                      2.4.2
pydantic_core                 2.10.1
pydub                         0.25.1
Pygments                      2.16.1
pygtrie                       2.5.0
pyparsing                     3.1.1
pypinyin                      0.44.0
pypinyin-dict                 0.6.0
pyreadline3                   3.4.1
pytest-runner                 6.0.0
python-dateutil               2.8.2
python-multipart              0.0.6
pytz                          2023.3.post1
pywin32                       306
pyworld                       0.2.12
PyYAML                        6.0.1
pyzmq                         25.1.1
rarfile                       4.1
referencing                   0.30.2
regex                         2023.10.3
requests                      2.31.0
requests-mock                 1.11.0
resampy                       0.2.2
rich                          13.6.0
rpds-py                       0.10.6
ruamel.yaml                   0.17.38
ruamel.yaml.clib              0.2.8
sacrebleu                     2.3.1
safetensors                   0.4.0
scikit-learn                  1.3.1
scipy                         1.11.3
semantic-version              2.10.0
sentencepiece                 0.1.99
seqeval                       1.2.2
setuptools                    68.0.0
six                           1.16.0
sniffio                       1.3.0
sortedcontainers              2.4.0
soundfile                     0.12.1
SpeechRecognition             3.10.0
stack-data                    0.6.3
starlette                     0.27.0
swig                          4.1.1
sympy                         1.12
tabulate                      0.9.0
TextGrid                      1.5
threadpoolctl                 3.2.0
tiktoken                      0.3.3
timer                         0.2.2
ToJyutping                    0.2.3
tokenizers                    0.13.3
toolz                         0.12.0
torch                         2.0.0a0+gitc6a572f
tornado                       6.3.3
tqdm                          4.66.1
traitlets                     5.11.2
trampoline                    0.1.2
transformers                  4.31.0
transformers-stream-generator 0.0.4
typeguard                     2.13.3
typer                         0.9.0
typing_extensions             4.8.0
tzdata                        2023.3
urllib3                       2.0.7
uvicorn                       0.23.2
visualdl                      2.4.2
Wave                          0.0.2
wcwidth                       0.2.8
webrtcvad                     2.0.10
websockets                    11.0.3
Werkzeug                      3.0.0
wheel                         0.41.2
win32-setctime                1.1.0
xxhash                        3.4.1
xyzservices                   2023.10.0
yacs                          0.1.8
yarl                          1.9.2
zhconv                        1.4.3
zhon                          2.0.2
zipp                          3.17.0
hkvision commented 10 months ago

Hi @KiwiHana

Seems we can't reproduce this issue on MTL iGPU (and also on Arc)

-------------------- Prompt --------------------
问:校长说:“校服上除了校徽别别别的” 这句话里校长表达什么意思?
答:
-------------------- Output --------------------
问:校长说:“校服上除了校徽别别别的” 这句话里校长表达什么意思?
答: 这句话的意思是除了校徽之外,校服上不能有别的标志。校长可能认为,校服是学校的象征,如果上面有其他的

Seems you are using an old version, could you please try to use the latest version instead?