Questions about the code

lxtGH / OMG-Seg

OMG-LLaVA and OMG-Seg codebase [CVPR-24 and NeurIPS-24]

Other

1.32k stars 50 forks source link

Questions about the code #24

Closed chunshengshi closed 3 months ago

chunshengshi commented 4 months ago

In the omg_llava_collate_fn.py file, there is a problem with attention_mask[:i] = True. I think it should be attention_mask[:,:i] = True. Hope that answers, thanks!

lxtGH commented 4 months ago

@zhang-tao-whu Please check this issue. It seems a bug from xtuner.

zhang-tao-whu commented 3 months ago

Thank you very much for your discovery. This is a bug from the xtuner library. After fixing this bug, we attempted retraining, and there was a little performance improvement. For example, the cIoU on refCOCO val without finetuning increased from 65.6 to 66.3.