[TensorRT-LLM] TensorRT-LLM version: 0.11.0.dev2024052800
2024-08-05 08:09:33.435 | INFO | llmc.utils.export_trtllm:cvt_trtllm_engine:93 - Start to export trtllm engine...
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████| 4/4 [00:02<00:00, 1.44it/s]
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/app/llmc/llmc/__main__.py", line 147, in <module>
main(config)
File "/app/llmc/llmc/__main__.py", line 88, in main
cvt_trtllm_engine(
File "/app/llmc/llmc/utils/export_trtllm.py", line 95, in cvt_trtllm_engine
convert_and_save_hf(hf_model, output_dir, cfg)
File "/app/llmc/llmc/utils/export_trtllm.py", line 88, in convert_and_save_hf
convert_and_save_rank(cfg, rank=0)
File "/app/llmc/llmc/utils/export_trtllm.py", line 75, in convert_and_save_rank
llama = LLaMAForCausalLM.from_hugging_face(
File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/models/llama/model.py", line 280, in from_hugging_face
llama = convert.from_hugging_face(
File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/models/llama/convert.py", line 1333, in from_hugging_face
llama.load(weights)
File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/models/modeling_utils.py", line 422, in load
raise RuntimeError(
RuntimeError: Required but not provided tensors:{'transformer.vocab_embedding.per_token_scale'}
Weights loaded. Total time: 00:01:54
Hello,
I'm tring to build with tensorrt, following is the config file:
I got following error: