Closed Bao0ne closed 1 month ago
it seems TPU is hang, could you check your libbmrt.so path is correct?
@zifeng-radxa I can run the Llama3 int4 provided in the documentation but it gives the above issue when trying to run the in8 model converted as per the documentation.
I found these files:
root@bm1684:/home/linaro# find / -name "libbmrt.so"
/opt/sophon/libsophon-0.4.9/lib/libbmrt.so
/data/LLM-TPU/models/ChatGLM3/support/lib_soc/libbmrt.so
/data/LLM-TPU/models/ChatGLM3/support/lib_pcie/libbmrt.so
/data/LLM-TPU/models/WizardCoder/demo/lib_soc/lib/libbmrt.so
/data/LLM-TPU/models/WizardCoder/demo/lib_pcie/lib/libbmrt.so
/data/LLM-TPU/models/ChatGLM2/support/lib_soc/libbmrt.so
/data/LLM-TPU/models/ChatGLM2/support/lib_pcie/libbmrt.so
/data/LLM-TPU/models/LWM/support/lib_soc/libbmrt.so
/data/LLM-TPU/models/LWM/support/lib_pcie/libbmrt.so
/data/LLM-TPU/models/Baichuan2/src/lib_soc/libbmrt.so
/data/LLM-TPU/models/Baichuan2/src/lib_pcie/libbmrt.so
/data/LLM-TPU/support/lib_soc/libbmrt.so
/data/LLM-TPU/support/lib_pcie/libbmrt.so
i think you need to use /data/LLM-TPU/support/lib_soc/libbmrt.so
or /data/LLM-TPU/support/lib_pcie/libbmrt.so
depend what kind of device you use
log: