[bug] 'MiniGPT4' object has no attribute 'chat_template'

[becauseofAI]

When run torchrun --nproc-per-node 1 train.py --cfg-path train_configs/minigpt4_stage2_finetune.yaml, I got the following error:

2023-10-18 23:40:35,805 [INFO] Building datasets...
2023-10-18 23:40:35,817 [INFO] Loading LLAMA
You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. This is expected, and simply means that the `legacy` (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set `legacy=False`. This should only be set if you understand what it means, and thouroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████| 2/2 [00:17<00:00,  8.70s/it]
2023-10-18 23:43:17,029 [INFO] Loading LLAMA Done
2023-10-18 23:43:17,029 [INFO] Loading VIT
2023-10-18 23:43:37,220 [INFO] freeze vision encoder
2023-10-18 23:43:37,220 [INFO] Loading VIT Done
Loading Q-Former
2023-10-18 23:43:40,244 [INFO] freeze Qformer
2023-10-18 23:43:41,130 [INFO] load checkpoint from /root/model/lavis/blip2/blip2_pretrained_flant5xxl.pth
Loading Q-Former Done
Load 4 training prompts
Prompt Example 
###Human: <Img><ImageHere></Img> Describe this image in detail. ###Assistant: 
Load MiniGPT-4 Checkpoint: /root/model/minigpt/pretrained_minigpt4_7b_stage1.pth
2023-10-18 23:43:41,257 [INFO] Start training
2023-10-18 23:43:44,337 [INFO] dataset_ratios not specified, datasets will be concatenated (map-style datasets) or chained (webdataset.DataPipeline).
2023-10-18 23:43:44,337 [INFO] Loaded 3439 records for train split from the dataset.
module.llama_proj.weight
module.llama_proj.bias
2023-10-18 23:43:44,353 [INFO] number of trainable parameters: 3149824
2023-10-18 23:43:44,353 [INFO] Start training epoch 0, 200 iters per inner epoch.
Traceback (most recent call last):
  File "train.py", line 103, in <module>
    main()
  File "train.py", line 99, in main
    runner.train()
  File "/root/MiniGPT-4/minigpt4/runners/runner_base.py", line 378, in train
    train_stats = self.train_epoch(cur_epoch)
  File "/root/MiniGPT-4/minigpt4/runners/runner_base.py", line 438, in train_epoch
    return self.task.train_epoch(
  File "/root/MiniGPT-4/minigpt4/tasks/base_task.py", line 114, in train_epoch
    return self._train_inner_loop(
  File "/root/MiniGPT-4/minigpt4/tasks/base_task.py", line 219, in _train_inner_loop
    loss = self.train_step(model=model, samples=samples)
  File "/root/MiniGPT-4/minigpt4/tasks/base_task.py", line 68, in train_step
    loss = model(samples)["loss"]
  File "/usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 1156, in forward
    output = self._run_ddp_forward(*inputs, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 1110, in _run_ddp_forward
    return module_to_run(*inputs[0], **kwargs[0])  # type: ignore[index]
  File "/usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/MiniGPT-4/minigpt4/models/minigpt_base.py", line 275, in forward
    self.preparing_embedding(samples)
  File "/root/MiniGPT-4/minigpt4/models/minigpt_base.py", line 238, in preparing_embedding
    if self.chat_template:
  File "/usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1614, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'MiniGPT4' object has no attribute 'chat_template'

[lkk117766]

I encountered the same problem in the first stage when using my own data.

[AlanXU07]

Have you resolved this issue? I have encountered the same problem when training the first stage using my own dataset.

[zhangwengyu999]

I have encountered the same problem when finetuning the second stage.

[TsuTikgiau]

Thank you for your interest! We have updated the code to fix this bug. Note that the training code you are using here is for the first version, namely MiniGPT-4. For the finetuning code of the second version, namely MiniGPT-v2, we will release it soon

[zhangwengyu999]

Thank you for your interest! We have updated the code to fix this bug. Note that the training code you are using here is for the first version, namely MiniGPT-4. For the finetuning code of the second version, namely MiniGPT-v2, we will release it soon

Thank you very much, problem solved.