CoderZhangYx
commented
9 months ago I found that CLIP tokeizer will automatically add one [start_token], but the other tokenizer i load did not do this, which is where the problem lies in
Closed CoderZhangYx closed 9 months ago
CoderZhangYx
commented
9 months ago I found that CLIP tokeizer will automatically add one [start_token], but the other tokenizer i load did not do this, which is where the problem lies in
Hi, thanks for your terrific work! During reading your code, I found that in https://github.com/dvlab-research/LISA/blob/7232bb8fa764f6d3a9b7928c975e26a67fde545c/model/LISA.py#L187 and in https://github.com/dvlab-research/LISA/blob/7232bb8fa764f6d3a9b7928c975e26a67fde545c/utils/dataset.py#L99, [START_TOKEN] (or [CLS_TOKEN] in some works) is consiered. However, the constructed text-prompt data seems to not have such a token. Did I ignore some related codes? Did i misunderstand your code? Any explanation would be helpful, thanks!