Start computing and quantizing weights... This may take a while.
Traceback (most recent call last):
File "~/mlc-llm/mlc_llm/build.py", line 42, in main
core.build_model_from_args(parsed_args)
File "~/mlc-llm/mlc_llm/core.py", line 619, in build_model_from_args
new_params = utils.convert_weights(param_manager, params, args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "~/mlc-llm/mlc_llm/utils.py", line 258, in convert_weights
vm["transform_params"]()
File "tvm/_ffi/_cython/./packed_func.pxi", line 332, in tvm._ffi._cy3.core.PackedFuncBase.__call__
File "tvm/_ffi/_cython/./packed_func.pxi", line 263, in tvm._ffi._cy3.core.FuncCall
File "tvm/_ffi/_cython/./packed_func.pxi", line 252, in tvm._ffi._cy3.core.FuncCall3
File "tvm/_ffi/_cython/./base.pxi", line 182, in tvm._ffi._cy3.core.CHECK_CALL
File "~/mambaforge/envs/mlc/lib/python3.11/site-packages/tvm/_ffi/base.py", line 476, in raise_last_ffi_error
raise py_err
File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback
File "~/mlc-llm/mlc_llm/relax_model/param_manager.py", line 558, in get_item
for torch_binname in [
^
File "~/mlc-llm/mlc_llm/relax_model/param_manager.py", line 559, in <listcomp>
self.torch_pname2binname[torch_pname] for torch_pname in torch_pnames
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
KeyError: 'model.layers.0.self_attn.q_proj.weight'
I quantized a custom fine-tuned llama2 70b model like this.
Then I updated
mlc_llm/quantization/__init__.py
like thisWhen I try to compile the model with mlc-llm,
I got this error.