about use image start and end token

[Hoteryoung]

In the finetune_lora.sh, the argument --mm_use_im_start_end is set to False. However, based on the paper (see figure below), it should be True. Furthermore, when I change this argument to True. The following error occurred:

Traceback (most recent call last):
  File "/xxx/Documents/code/geochat/geochat/train/train_mem.py", line 13, in <module>
    train()
  File "/xxx/Documents/code/geochat/geochat/train/train.py", line 952, in train
    model.initialize_vision_tokenizer(model_args, tokenizer=tokenizer)
  File "/xxx/Documents/code/geochat/geochat/model/geochat_arch.py", line 343, in initialize_vision_tokenizer
    embed_tokens_weight = mm_projector_weights["model.embed_tokens.weight"]
KeyError: 'model.embed_tokens.weight'

[732259408]

@Hoteryoung , do you resolve this issue?

[Hoteryoung]

Not yet. I just keep it to be False