search

mlpc-ucsd / BLIVA

(AAAI 2024) BLIVA: A Simple Multimodal LLM for Better Handling of Text-rich Visual Questions
https://arxiv.org/abs/2308.09936
BSD 3-Clause "New" or "Revised" License
258 stars 26 forks source link

The evaluation performance on the Filckr30k dataset #27

Open mingtouyizu opened 1 month ago

mingtouyizu commented 1 month ago

Why do I only get a CIDEr score of 0.065 on Flickr30k for bliva_vicuna7b, even if multiplied by 10 it's only 0.65? Could you tell me what might have gone wrong in this process?? Thanks. {"test": {"Bleu_1": 0.23795275042621625, "Bleu_2": 0.15775161595849657, "Bleu_3": 0.1035576186878422, "Bleu_4": 0.06837100713764228, "METEOR": 0.19554874963685032, "ROUGE_L": 0.276937750791617, "CIDEr": 0.06517325574578513, "SPICE": 0.17977547067526298}}

jjt076 commented 1 month ago

Why do I only get a CIDEr score of 0.065 on Flickr30k for bliva_vicuna7b, even if multiplied by 10 it's only 0.65? Could you tell me what might have gone wrong in this process?? Thanks. {"test": {"Bleu_1": 0.23795275042621625, "Bleu_2": 0.15775161595849657, "Bleu_3": 0.1035576186878422, "Bleu_4": 0.06837100713764228, "METEOR": 0.19554874963685032, "ROUGE_L": 0.276937750791617, "CIDEr": 0.06517325574578513, "SPICE": 0.17977547067526298}}

Can you tell me where the test code is?