When I use 7B_FT_float16
python MinimumExample/Example_ONNX_LlamaV2.py --onnx_file 7B_FT_float16/ONNX/LlamaV2_7B_FT_float16.onnx --embedding_file 7B_FT_float16/embeddings.pth --tokenizer_path tokenizer.model --prompt "What is the lightest element?"
The response is as expected:
The lightest element is hydrogen. Hydrogen is the lightest element on the periodic table, with an atomic mass of 1.00794 u (unified atomic mass units).
When I use 7B_float16
python MinimumExample/Example_ONNX_LlamaV2.py --onnx_file 7B_float16/ONNX/LlamaV2_7B_float16.onnx --embedding_file 7B_float16/embeddings.pth --tokenizer_path tokenizer.model --prompt "What is the lightest element?"
The response is strange:
What is the lightest element on the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is the lightest element in the periodic table?
What is the lightest element in the universe?
What is
When I use 7B_FT_float16
python MinimumExample/Example_ONNX_LlamaV2.py --onnx_file 7B_FT_float16/ONNX/LlamaV2_7B_FT_float16.onnx --embedding_file 7B_FT_float16/embeddings.pth --tokenizer_path tokenizer.model --prompt "What is the lightest element?"
The response is as expected:The lightest element is hydrogen. Hydrogen is the lightest element on the periodic table, with an atomic mass of 1.00794 u (unified atomic mass units).
When I use 7B_float16
python MinimumExample/Example_ONNX_LlamaV2.py --onnx_file 7B_float16/ONNX/LlamaV2_7B_float16.onnx --embedding_file 7B_float16/embeddings.pth --tokenizer_path tokenizer.model --prompt "What is the lightest element?"
The response is strange: