Closed na20215 closed 8 months ago
Hi @na20215, I have just tested it with QNN 2.19 and it works fine. Could you please try QNN < 2.20 ?
Hi @na20215, Thanks for pointing out. This error is not matter about llama2 compilation. We lost to handle compile_only argument in dummy_llama2.py. I have created a PR to fix it.
Hi @na20215, I have just tested it with QNN 2.19 and it works fine. Could you please try QNN < 2.20 ?
Hi @salykovaa thanks for replying I turn the qnn version from 2.20.0.240223 to 2.19.4.240226, and try to run the commend again:
python -m executorch.examples.qualcomm.scripts.dummy_llama2 -b executorch/build_android -m SM8550 --compile_only
but I got more error message like this:
[WARNING] [Qnn ExecuTorch]: Qnn Backend op validation failed with error: 3110
[QNN Partitioner Op Support]: aten.mul.Tensor | False
[ERROR] [Qnn ExecuTorch]: Input[1] has incorrect Rank 0.
[ERROR] [Qnn ExecuTorch]: Input[1] has incorrect Rank 0.
[ERROR] [Qnn ExecuTorch]: Op specific validation failed.
[ERROR] [Qnn ExecuTorch]: QnnDsp <E> validateNativeOps master op validator aten_mul_tensor_91:qti.aisw:ElementWiseMultiply failed 3110
[ERROR] [Qnn ExecuTorch]: QnnDsp <E> QnnBackend_validateOpConfig failed 3110
[ERROR] [Qnn ExecuTorch]: QnnDsp <E> Failed to validate op aten_mul_tensor_91 with error 0xc26
The original error still persists. It seems that the QNN version is partly related to my issue. Could you please share the QNN version which you are using? Thanks again!
Hi @na20215, Thanks for pointing out. This error is not matter about llama2 compilation. We lost to handle compile_only argument in dummy_llama2.py. I have created a PR to fix it.
thanks for replying. I would like to inquire whether the current version of executorch supports compiling the llama2-7b model for the qnn backend. If it does not, is it possible to achieve this functionality by modifying the dummy llama or other parts of the code? I am very interested in this feature. Would this task be very difficult?
@na20215 sorry, previously I run the script without --compile_only
flag, therefore it worked. Both qnn versions 2.19 and 2.20 work without --compile_only
flag.
python -m executorch.examples.qualcomm.scripts.dummy_llama2 -b executorch/build_android -m SM8550 --device SERIAL_NUMBER
thanks for replying. I would like to inquire whether the current version of executorch supports compiling the llama2-7b model for the qnn backend. If it does not, is it possible to achieve this functionality by modifying the dummy llama or other parts of the code? I am very interested in this feature. Would this task be very difficult?
Thanks for patient. This example is to show the feasibility of exporting Llama2 model with Qualcomm backend in ExecuTorch. For more details, you could refer to here. We are working on llama2-7b model and stay tune on it.
Hello, When I attempted to convert the default provided llama model using the command:
I encountered an error.
I was wondering if this issue could possibly be related to my use of qnn2.20? Any advice or suggestions you could provide would be greatly appreciated.