search

microsoft / Llama-2-Onnx

Other
1.01k stars 91 forks source link

Model inference accuracy #43

Open realhaik opened 8 months ago

realhaik commented 8 months ago

Running this on RTX4090

C:\AI\Llama2-Onnx\Llama-2-Onnx>python MinimumExample/Example_ONNX_LlamaV2.py --onnx_file 7B_float16/ONNX/LlamaV2_7B_float16.onnx --embedding_file 7B_float16/embeddings.pth --tokenizer_path tokenizer.model --prompt "What is the lightest element?" C:\Users\Real\AppData\Roaming\Python\Python310\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'DmlExecutionProvider' is not in available provider names.Available providers: 'TensorrtExecutionProvider, CUDAExecutionProvider, CPUExecutionProvider' warnings.warn(

What is the lightest element on the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is

C:\AI\Llama2-Onnx\Llama-2-Onnx> C:\AI\Llama2-Onnx\Llama-2-Onnx>python MinimumExample/Example_ONNX_LlamaV2.py --onnx_file 7B_float16/ONNX/LlamaV2_7B_float16.onnx --embedding_file 7B_float16/embeddings.pth --tokenizer_path tokenizer.model --prompt "What is the lightest element?" C:\Users\Real\AppData\Roaming\Python\Python310\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'DmlExecutionProvider' is not in available provider names.Available providers: 'TensorrtExecutionProvider, CUDAExecutionProvider, CPUExecutionProvider' warnings.warn(

What is the lightest element on the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is the lightest element in the periodic table? What is the lightest element in the universe? What is

C:\AI\Llama2-Onnx\Llama-2-Onnx>python MinimumExample/Example_ONNX_LlamaV2.py --onnx_file 7B_float16/ONNX/LlamaV2_7B_float16.onnx --embedding_file 7B_float16/embeddings.pth --tokenizer_path tokenizer.model --prompt "What is the lightest element on the periodic table" C:\Users\Real\AppData\Roaming\Python\Python310\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'DmlExecutionProvider' is not in available provider names.Available providers: 'TensorrtExecutionProvider, CUDAExecutionProvider, CPUExecutionProvider' warnings.warn( ? What is the lightest element on the periodic table? Hydrogen is the lightest element on the periodic table. It is also the most abundant element in the universe. Hydrogen is a gas at room temperature and pressure. What is the lightest element in the periodic table? What is the lightest element in the periodic table? Hydrogen is the lightest element in the periodic table. It is also the most abundant element in the universe. Hydrogen is a gas at room temperature and pressure. What is the lightest element in the periodic table? Hydrogen is the lightest element in the periodic table. It is also the most abundant element in the universe. Hydrogen is a gas at room temperature and pressure. What is the lightest element in the periodic table? Hydrogen is the lightest element in the periodic table. It is also the most abundant element in the universe. Hydrogen is a gas at room temperature and pressure. Hydrogen is the lightest element in the periodic table. It is also the most abundant element in the universe. Hydrogen is a gas at room temperature and pressure. What is the lightest element in the periodic

C:\AI\Llama2-Onnx\Llama-2-Onnx>

mszhanyi commented 7 months ago

I came acorss the same issue. But it works if you use 7B_FT_Float16 python MinimumExample/Example_ONNX_LlamaV2.py --onnx_file 7B_FT_float16/ONNX/LlamaV2_7B_FT_float16.onnx --embedding_file 7B_FT_float16/embeddings.pth --tokenizer_path tokenizer.model --prompt "What is the lightest element?"