search

microsoft / onnxruntime-genai

Generative AI extensions for onnxruntime
MIT License
418 stars 95 forks source link

Building from Source for Jetson #818

Open SRai22 opened 3 weeks ago

SRai22 commented 3 weeks ago

Hi,

I'm trying to build onnxruntime-genai with cuda support on Jetson device. I have onnxruntime-1.18.0 and onnxruntime-gpu 1.18.0.

I have placed the .so files in ort/lib/ and .h files in ort/include as instructed.

But when building, it fetches onnxruntime version 1.17.1 into the _deps. Does this version have to match the onnxruntime I have installed on the machine ? I'm not able to use CUDA to run genai.

Kindly help me on how to change the onnxruntime source version getting fetched by cmake to customizable option.

Thanks in advance.

PredyDaddy commented 3 weeks ago

Hi,

I'm trying to build onnxruntime-genai with cuda support on Jetson device. I have onnxruntime-1.18.0 and onnxruntime-gpu 1.18.0.

I have placed the .so files in ort/lib/ and .h files in ort/include as instructed.

But when building, it fetches onnxruntime version 1.17.1 into the _deps. Does this version have to match the onnxruntime I have installed on the machine ? I'm not able to use CUDA to run genai.

Kindly help me on how to change the onnxruntime source version getting fetched by cmake to customizable option.

Thanks in advance.

sample problem here

natke commented 2 weeks ago

Can you share the logs of the build failure here please? @SRai22

SRai22 commented 4 days ago

The build does not fail. It completes and generates a wheel. But when running, the library does not provide the GPU provider.