microsoft / onnxruntime-genai

Generative AI extensions for onnxruntime
MIT License
513 stars 127 forks source link

Mac build from source not working: onnx .dylib link [Mac M3/ARM/MACOS] #909

Open farris opened 1 month ago

farris commented 1 month ago

Describe the bug When following the steps to build the generate() API from source link, there is an issue linking the API back to the onnxruntime dependency

Traceback (most recent call last):
  File "onnxruntime-genai/examples/python/phi3-qa.py", line 1, in <module>
    import onnxruntime_genai as og
  File "/.venvs/universal_venv/lib/python3.12/site-packages/onnxruntime_genai/__init__.py", line 9, in <module>
    _dll_directory.add_onnxruntime_dependency(__id__)
  File "/.venvs/universal_venv/lib/python3.12/site-packages/onnxruntime_genai/_dll_directory.py", line 62, in add_onnxruntime_dependency
    raise ImportError("Could not find the onnxruntime shared library.")
ImportError: Could not find the onnxruntime shared library.

Expected behavior To fix it, I changed the _dll_directory script to point to the onnx dylib

target_lib_path = "~/Desktop/dev/onnxruntime-genai/ort/lib/libonnxruntime.dylib"

Additional context Do with this info what you will 😄

skyline75489 commented 1 month ago

Hi @farris, the current version of GenAI doesn't have full support on macOS. I'm working on improving this and it should be improved in the next release. Thanks for trying it out :)