microsoft / T-MAC

Low-bit LLM inference on CPU with lookup table
MIT License
420 stars 32 forks source link

codegen Error in cross-compiling Android on Mac #9

Closed zk1998 closed 1 month ago

zk1998 commented 1 month ago

Hi @kaleid-liner ,Thanks for your works!

zk1998 commented 1 month ago

I am reproducing T-mac on Android.I have set up an RPC service and successfully passed the test in tvm/apps/android_rpc/tests/android_rpc_test.py. when I use python compile.py -o tuned_android -d android -t -tb -gc -da -nt 1 -gs 128 -ags 64 -m hf-bitnet-3, the following error occurred:

File "/Users/tmp/tmp/T-MAC/3rdparty/tvm/python/tvm/autotvm/measure/measure_methods.py", line 568, in call func, arg_info = _build_func_common(measure_input, self.runtime, **kwargs) File "/Users/tmp/tmp/T-MAC/3rdparty/tvm/python/tvm/autotvm/measure/measure_methods.py", line 519, in _build_func_common func = build(s, args, target=target, runtime=runtime) File "/Users/tmp/tmp/T-MAC/3rdparty/tvm/python/tvm/driver/build_module.py", line 299, in build rt_mod_host = _driver_ffi.tir_to_runtime(annotated_mods, target_host) File "/Users/tmp/tmp/T-MAC/3rdparty/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 245, in call raise_last_ffi_error() File "/Users/tmp/tmp/T-MAC/3rdparty/tvm/python/tvm/_ffi/base.py", line 481, in raise_last_ffi_error raise py_err tvm._ffi.base.TVMError: Traceback (most recent call last): File "/Users/tmp/tmp/T-MAC/3rdparty/tvm/src/target/llvm/codegen_llvm.cc", line 354 TVMError: LLVM module verification failed with the following errors: Declaration may not be in a Comdat! ptr @_Z29tbl_g4_int8_float_update_implILb1ELi16ELi2ELi16ELb0ELb0ELb1EEiiPDhPaPhS0_S0S0

The codegen does not produce this error on M1, and I am not very familiar with TVM. Could you provide a solution or debugging suggestions?

kaleid-liner commented 1 month ago

I have pushed a fix. The issue is caused by failure of template inlining, and LLVM thus can't find the symbol.

zk1998 commented 1 month ago

Thanks, it works!