search

salesforce / LAVIS

LAVIS - A One-stop Library for Language-Vision Intelligence
BSD 3-Clause "New" or "Revised" License
9.6k stars 942 forks source link

Is it a writting Error about bos_token in instuctblip? #507

Open Coobiw opened 1 year ago

Coobiw commented 1 year ago

Hello, thanks to your great work!

In blip2_vicuna_instruct.py, the bos_token of LLM has been changed. Originally, it is '< s >' with idx:1. But after the following code:

self.llm_tokenizer.add_special_tokens({'pad_token': '[PAD]'})
self.llm_tokenizer.add_special_tokens({'bos_token': '</s>'})
self.llm_tokenizer.add_special_tokens({'eos_token': '</s>'})
self.llm_tokenizer.add_special_tokens({'unk_token': '</s>'})

After this ops, the bos_token is changed into '< /s >' with idx:2, the same as the eos_token.

Is it a writting Error? Had you used this code for instruction tuning? If used, will it lead some bad influences?

Thanks for your reply!

xianzhangzx commented 5 months ago

hello, i find it too. Have you solved the problem?

Coobiw commented 5 months ago

It's been a long time. I remember that I've modify the token_id setting in generate function of InstructBLIP. Actually, this doesn't have much impact at all.