search

microsoft / onnxruntime-genai

Generative AI extensions for onnxruntime
MIT License
502 stars 126 forks source link

[Documentation] Not able to Install phi3-vision #651

Closed GauravKatariya closed 4 months ago

GauravKatariya commented 4 months ago

Describe the documentation issue

I am trying to follow the instructions https://onnxruntime.ai/docs/genai/tutorials/phi3-v.html. I am trying Run with NVIDIA CUDA approach.

I am able to setup the CUDA image image

I am able to download the model image

I am trying to run the code of https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/phi3v.py -o phi3v.py

and getting below error D:\POC\Fresh-phi3>python phi3v.py -m cuda-int4-rtn-block-32 Loading model... Traceback (most recent call last): File "D:\POC\Fresh-phi3\phi3v.py", line 63, in run(args) File "D:\POC\Fresh-phi3\phi3v.py", line 13, in run model = og.Model(args.model_path) ^^^^^^^^^^^^^^^^^^^^^^^^^ onnxruntime_genai.onnxruntime_genai.OrtException: C:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1426 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\gakatari\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\onnxruntime_genai\onnxruntime_providers_cuda.dll"

Page / URL

No response

natke commented 4 months ago

Hi @GauravKatariya, can you list your installed Python packages please?

baijumeswani commented 4 months ago

@GauravKatariya can you please check your environment variables? In particular, the CUDA_PATH environment system variable should point to the path where cuda 11.8 is installed. For me, it is: 

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8

image

baijumeswani commented 4 months ago

I hope the information provided above helped. I'll close this issue now. Please re-open in case you're still encountering problems.

GauravKatariya commented 4 months ago

@baijumeswani image (3) image

GauravKatariya commented 4 months ago

@natke

certifi 2024.6.2 charset-normalizer 3.3.2 colorama 0.4.6 filelock 3.15.4 fsspec 2024.6.0 huggingface-hub 0.23.4 idna 3.7 inquirerpy 0.3.4 intel-openmp 2021.4.0 Jinja2 3.1.3 MarkupSafe 2.1.5 mkl 2021.4.0 mpmath 1.3.0 networkx 3.2.1 numpy 2.0.0 onnxruntime-genai-cuda 0.3.0 packaging 24.1 pfzy 0.3.4 pillow 10.2.0 pip 24.1.1 prompt_toolkit 3.0.47 pyreadline3 3.4.1 PyYAML 6.0.1 requests 2.32.3 setuptools 65.5.0 sympy 1.12 tbb 2021.11.0 torch 2.3.1+cu118 torchaudio 2.3.1+cu118 torchvision 0.18.1+cu118 tqdm 4.66.4 typing_extensions 4.12.2 urllib3 2.2.2 wcwidth 0.2.13

Replikate648 commented 3 months ago

I am getting the same issue when running the command "python phi3v.py -m cuda-int4-rtn-block-32" from my anaconda env. Error logs:

"Loading model... Traceback (most recent call last): File "phi3v.py", line 66, in run(args) File "phi3v.py", line 16, in run model = og.Model(args.model_path) ^^^^^^^^^^^^^^^^^^^^^^^^^ onnxruntime_genai.onnxruntime_genai.OrtException: D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1426 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "onnxruntime_providers_cuda.dll""