microsoft / T-MAC

Low-bit LLM inference on CPU with lookup table
MIT License
420 stars 32 forks source link

question about run_pipeline.py #22

Closed sunj0104 closed 3 weeks ago

sunj0104 commented 3 weeks ago

hello, I run our code on windows and vs2022 follow the guild. when I run the run_pipeline.py, I got the wrong result in log file. Do you know how to fix it, thank you very much~The log file content as follow:

[Task qgemm_lut_t4_int8_m8192_k4096_n1_b2] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/4) | 0.00 s [Task qgemm_lut_t4_int8_m8192_k4096_n1_b2] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (4/4) | 22.26 sWARNING:root:Could not find any valid schedule for task Task(func_name=qgemm_lut_t4_int8_m8192_k4096_n1_b2, args=(8192, 1, 4096), kwargs={}, workload=('qgemm_lut_t4_int8_m8192_k4096_n1_b2', 8192, 1, 4096)). A file containing the errors has been written to C:\Users\sunji\AppData\Local\Temp\tvm_tuning_errors_6pbx8a0j.log. WARNING:autotvm:Cannot find config for target=llvm -keys=cpu -mcpu=core-avx2 -mtriple=x86_64-pc-windows-msvc, workload=('qgemm_lut_t4_int8_m8192_k4096_n1_b2', 8192, 1, 4096). A fallback configuration is used, which may bring great performance regression. Done. Traceback (most recent call last): File "compile.py", line 240, in main() File "compile.py", line 230, in main compile(**device_kwargs) File "compile.py", line 126, in compile qgemm_mod = qgemm_lut.compile( File "C:\Users\sunji\anaconda3\envs\tvm-build\lib\site-packages\t_mac\ops\base.py", line 268, in compile func = tvm.build(s, tensors, name=template_name) File "C:\tech_projects\T-MAC\3rdparty\tvm\python\tvm\driver\build_module.py", line 297, in build rt_mod_host = _driver_ffi.tir_to_runtime(annotated_mods, target_host) File "C:\tech_projects\T-MAC\3rdparty\tvm\python\tvm_ffi_ctypes\packed_func.py", line 239, in call raise_last_ffi_error() File "C:\tech_projects\T-MAC\3rdparty\tvm\python\tvm_ffi\base.py", line 481, in raise_last_ffi_error raise py_err tvm._ffi.base.TVMError: Traceback (most recent call last): %42 = zext nneg i32 %41 to i64 ^ File "C:\tech_projects\T-MAC\3rdparty\tvm\src\target\llvm\llvm_instance.cc", line 176 error: expected type

kaleid-liner commented 3 weeks ago

Can you provide the version of clang and llvm-config?

sunj0104 commented 3 weeks ago

Sure,thanks for your quick response~ clang version 18.1.8; llvm-config version 17.0.6

kaleid-liner commented 3 weeks ago

TVM is incompatible with clang 18 or llvm 18. You can use clang=17.0.6 instead.

sunj0104 commented 3 weeks ago

great, it is normal now~thank you again

kaleid-liner commented 3 weeks ago

I will close the issue as resolved