search

tylin / coco-caption

Other
1.12k stars 545 forks source link

How to calculate the spice scores in other languages #57

Open ltp1995 opened 3 years ago

ltp1995 commented 3 years ago

The code seems only support spice evaluation in English, how can i obtain the spice scores in languages such as Chinese, Japanese, and Spanish.

jontooy commented 2 years ago

Hi,

I'm having the same question. I want to do SPICE evaluation on Arabic Captions specifically. In the 'spice-1.0.jar' file there is a README file that says:

Dependencies

Since the the version of Stanford CoreNLP is not compatible with Arabic, I wonder if I can:

  1. Replace Stanford Scene Graph Parser with the latest version, updated to work with CoreNLP 4.2.0
  2. Replace CoreNLP 3.6.0 with 4.2.0 with Arabic models

I actually tried to download and replace these files in the COCO-caption API, but with no improved SPICE scores...

Do you think this task is possible or no?

mary-0830 commented 7 months ago

Is there any other library for evaluating captions in other languages? @jontooy @ltp1995