search

quic / ai-hub-models

The Qualcomm® AI Hub Models are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.
https://aihub.qualcomm.com
BSD 3-Clause "New" or "Revised" License
438 stars 60 forks source link

ModuleNotFoundError: No module named 'libPyIrGraph' #97

Open Harsha0056 opened 2 weeks ago

Harsha0056 commented 2 weeks ago

Describe the bug when trying to run python gen_ondevice_llama.py --hub-model-id mrmdw7lwq,mjqyx81rm,m1q8748pm,m0q92582m,m9m551gym,m7qk5wy7q,m1m60802m,mwn0k9zzq --output-dir ./export --tokenizer-zip-path ./tokenizer.zip --target-gen snapdragon-gen2 --target-os windows

On a docker machine with base image Ubuntu 22.04 and python 3.10 on HP Elitebook ultra laptop with processor Snapdragon(TM) X Elite - X1E78100 - Qualcomm(TM) Oryon(TM) CPU 3.42 GHz

Error Using previously extracted model Generating model lib for split-0 pp ./export/intermediate_data/input_data/data_pp_0.h5 export/intermediate_data/input_data/data_pp_0/_data /opt/qualcomm_ai_engine_direct.2.25.0.240728.Linux-AnyCPU/bin/x86_64-linux-clang/qnn-onnx-converter --input_network ./export/intermediate_data/input_models/model_pp_0/Llama2_PromptProcessor_1_Quantized.aimet/Llama2_PromptProcessor_1_Quantized.onnx --output_path ./export/intermediate_data/cpp_models/Llama2_PromptProcessor_1_Quantized.cpp --no_simplification --quantization_overrides ./export/intermediate_data/input_models/model_pp_0/Llama2_PromptProcessor_1_Quantized.aimet/Llama2_PromptProcessor_1_Quantized.encodings --preserve_io layout input_ids attention_mask position_ids_cos position_ids_sin layers_7_add_out_0 past_key_0_out past_value_0_out past_key_1_out past_value_1_out past_key_2_out past_value_2_out past_key_3_out past_value_3_out past_key_4_out past_value_4_out past_key_5_out past_value_5_out past_key_6_out past_value_6_out past_key_7_out past_value_7_out --input_layout attention_mask NONTRIVIAL --input_layout position_ids_cos NONTRIVIAL --input_layout position_ids_sin NONTRIVIAL --input_list export/intermediate_data/input_data/data_pp_0/_data/input_list.txt --input_dim input_ids 1,1024 --input_dim attention_mask 1,1,1024,1024 --input_dim position_ids_cos 1,1,1024,64 --input_dim position_ids_sin 1,1,1024,64 --input_dtype input_ids int32 --input_dtype attention_mask float32 --input_dtype position_ids_cos float32 --input_dtype position_ids_sin float32 --bias_bitwidth 32 --weight_bw 8 --act_bitwidth 16 Traceback (most recent call last): File "/opt/qualcomm_ai_engine_direct.2.25.0.240728.Linux-AnyCPU/lib/python/qti/aisw/converters/common/init.py", line 85, in from . import libPyIrGraph as ir_graph ImportError: cannot import name 'libPyIrGraph' from partially initialized module 'qti.aisw.converters.common' (most likely due to a circular import) (/opt/qualcomm_ai_engine_direct.2.25.0.240728.Linux-AnyCPU/lib/python/qti/aisw/converters/common/init.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/qualcomm_ai_engine_direct.2.25.0.240728.Linux-AnyCPU/bin/x86_64-linux-clang/qnn-onnx-converter", line 12, in from qti.aisw.converters import onnx as onnx_frontend File "/opt/qualcomm_ai_engine_direct.2.25.0.240728.Linux-AnyCPU/lib/python/qti/aisw/converters/onnx/init.py", line 8, in from .onnx_to_ir import OnnxConverterFrontend File "/opt/qualcomm_ai_engine_direct.2.25.0.240728.Linux-AnyCPU/lib/python/qti/aisw/converters/onnx/onnx_to_ir.py", line 14, in from qti.aisw.converters.common.utils import code_to_message File "/opt/qualcomm_ai_engine_direct.2.25.0.240728.Linux-AnyCPU/lib/python/qti/aisw/converters/common/init.py", line 95, in raise e File "/opt/qualcomm_ai_engine_direct.2.25.0.240728.Linux-AnyCPU/lib/python/qti/aisw/converters/common/init.py", line 53, in import libPyIrGraph as ir_graph ModuleNotFoundError: No module named 'libPyIrGraph' Generating model lib... /opt/qualcomm_ai_engine_direct.2.25.0.240728.Linux-AnyCPU/bin/x86_64-linux-clang/qnn-model-lib-generator -c ./export/intermediate_data/cpp_models/Llama2_PromptProcessor_1_Quantized.cpp -b ./export/intermediate_data/cpp_models/Llama2_PromptProcessor_1_Quantized.bin -t x86_64-linux-clang -o ./export/intermediate_data/model_libs -l Llama2_PromptProcessor_1_Quantized 2024-09-13 14:15:20,021 - INFO - qnn-model-lib-generator: Model cpp file path : export/intermediate_data/cpp_models/Llama2_PromptProcessor_1_Quantized.cpp 2024-09-13 14:15:20,021 - INFO - qnn-model-lib-generator: Model bin file path : export/intermediate_data/cpp_models/Llama2_PromptProcessor_1_Quantized.bin 2024-09-13 14:15:20,021 - INFO - qnn-model-lib-generator: Library target : [['x86_64-linux-clang']] 2024-09-13 14:15:20,022 - INFO - qnn-model-lib-generator: Library name : Llama2_PromptProcessor_1_Quantized 2024-09-13 14:15:20,022 - INFO - qnn-model-lib-generator: Output directory : export/intermediate_data/model_libs Traceback (most recent call last): File "/opt/qualcomm_ai_engine_direct.2.25.0.240728.Linux-AnyCPU/bin/x86_64-linux-clang/qnn-model-lib-generator", line 495, in main() File "/opt/qualcomm_ai_engine_direct.2.25.0.240728.Linux-AnyCPU/bin/x86_64-linux-clang/qnn-model-lib-generator", line 489, in main result = generator.build_targets(config) File "/opt/qualcomm_ai_engine_direct.2.25.0.240728.Linux-AnyCPU/bin/x86_64-linux-clang/qnn-model-lib-generator", line 393, in build_targets self._normalize_config(config) File "/opt/qualcomm_ai_engine_direct.2.25.0.240728.Linux-AnyCPU/bin/x86_64-linux-clang/qnn-model-lib-generator", line 361, in _normalize_config raise FileNotFoundError(f'Unable to find the model source file, invalid path: {config.model_cpp.absolute()}') FileNotFoundError: Unable to find the model source file, invalid path: /opt/export/intermediate_data/cpp_models/Llama2_PromptProcessor_1_Quantized.cpp Traceback (most recent call last): File "/opt/gen_ondevice_llama.py", line 67, in main() File "/opt/gen_ondevice_llama.py", line 61, in main generate_shared_bins( File "/opt/utils.py", line 546, in generate_shared_bins generate_lib( File "/opt/utils.py", line 276, in generate_lib raise RuntimeError("The QNN graph compiler did not produce the output file") RuntimeError: The QNN graph compiler did not produce the output file

Host configuration:

Additional context Device Specs

Processor: Snapdragon(TM) X Elite - X1E78100 - Qualcomm(TM) Oryon(TM) CPU 3.42 GHz which has both GPU and NPU

bhushan23 commented 2 weeks ago

Hi @Harsha0056 qnn python tools require python3.10 see https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-2/setup.html?product=1601111740010412

Could you please try install python310 for qnn virtualenv?

quic-rneti commented 1 week ago

Please make sure you are using Ubuntu amd version instead of ubuntu arm64.

quic-rneti commented 1 week ago

Dear User - as suggested above, this is purely a QNN SDK setup issue. For any AI SDK issues, please post your questions at https://mysupport.qualcomm.com/supportforums/s/ (AI SDK & Tools)

AI Hub slack, and AI Hub github are intended for supporting AI Hub issues, not AI SDK issues.

quic-rneti commented 1 week ago

Dear User - Since you are running on X-Elite laptop, you can use windows qnn sdk for all conversion, quantization, and execution.

Also, when you are installing python on X-Elite, you need to select ARM64 variants.

For any further assistance on QNN SDK setup or any AI SDK related issues, please post at https://mysupport.qualcomm.com/supportforums/s/ (AI SDK & Tools)

Harsha0056 commented 6 days ago

when i ran into this error , i even tried in Ubuntu 22.04 , Linux x 86 Machine , with python 3.10 env [ As these specs were mentioned in the repo] and i faced the same issue QNN

YLlllllllllll commented 2 days ago

I have the same issue. Although the environment and Python version conform to the instructions, it keeps saying 'circular import' no matter how I reset the environment. image

bhushan23 commented 13 hours ago

HI @YLlllllllllll root cause is missing libpython3.10.so

you can install these package with the following command

sudo apt-get install python3-dev

circular dependency issue is coming from partially imported object which is result of this missing so

bhushan23 commented 10 hours ago

when i ran into this error , i even tried in Ubuntu 22.04 , Linux x 86 Machine , with python 3.10 env [ As these specs were mentioned in the repo] and i faced the same issue QNN

Hi @Harsha0056, as you can see there's an error with libc++ as a root cause for this partial import.

Coudl you please install clang / libc++? sometimes you have to ensure these libraries are discoverable by build tools