Problems with computing SPICE score.

[paniquex]

Hi

I tried to solve it by myself, but it was unsuccessful

OS: Ubuntu 18.04 LTS

java -version output:

openjdk version "1.8.0_242"
OpenJDK Runtime Environment (build 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08)
OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)

Error message:

paniquex, [06.03.20 20:58]
computing SPICE score...
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: org/json/simple/parser/ParseException
  at java.lang.Class.getDeclaredMethods0(Native Method)
  at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
  at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
  at java.lang.Class.getMethod0(Class.java:3018)
  at java.lang.Class.getMethod(Class.java:1784)
  at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
  at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: org.json.simple.parser.ParseException
  at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
  ... 7 more
---------------------------------------------------------------------------
CalledProcessError                        Traceback (most recent call last)
~/Documents/Automated_Audio_Captioning_DCASE2020/dcase-2020-baseline/main.py in <module>
     56 
     57 if name == 'main':
---> 58     main()
     59 
     60 # EOF

~/Documents/Automated_Audio_Captioning_DCASE2020/dcase-2020-baseline/main.py in main()
     52     if settings['workflow']['dnn_training'] or \
     53             settings['workflow']['dnn_evaluation']:
---> 54         method.method(settings)
     55 
     56 

~/Documents/Automated_Audio_Captioning_DCASE2020/dcase-2020-baseline/processes/method.py in method(settings)
    451             settings_data=settings['dnn_training_settings']['data'],
    452             settings_io=settings['dirs_and_files'],
--> 453             indices_list=indices_list)
    454         logger_inner.info('Evaluation done')
    455 

~/Documents/Automated_Audio_Captioning_DCASE2020/dcase-2020-baseline/processes/method.py in _do_evaluation(model, settings_data, settings_io, indices_list)
    177     logger_main.info('Evaluation done')
    178 
--> 179     metrics = evaluate_metrics(captions_pred, captions_gt)
    180 
    181     for metric, values in metrics.items():

~/Documents/Automated_Audio_Captioning_DCASE2020/dcase-2020-baseline/eval_metrics.py in evaluate_metrics(prediction_file, reference_file, nb_reference_captions)
    287         ground_truths.append([reference_dict[file_name][cap] for cap in cap_names])
    288 
--> 289     metrics, per_file_metrics = evaluate_metrics_from_lists(predictions, ground_truths)
    290 
    291     total_metrics = combine_single_and_per_file_metrics(

~/Documents/Automated_Audio_Captioning_DCASE2020/dcase-2020-baseline/eval_metrics.py in evaluate_metrics_from_lists(predictions, ground_truths, ids)
    159     write_json(pred, pred_file)
    160 
--> 161     metrics, per_file_metrics = evaluate_metrics_from_files(pred_file, ref_file)
    162 
    163     # Delete temporary files

~/Documents/Automated_Audio_Captioning_DCASE2020/dcase-2020-baseline/eval_metrics.py in evaluate_metrics_from_files(pred_file, ref_file)
    108     cocoEval = COCOEvalCap(coco, cocoRes)
    109     cocoEval.params['audio_id'] = cocoRes.getAudioIds()
--> 110     cocoEval.evaluate()
    111 
    112     # Make dict from metrics

~/Documents/Automated_Audio_Captioning_DCASE2020/dcase-2020-baseline/coco_caption/pycocoevalcap/eval.py in evaluate(self, verbose)
     60             if verbose:
     61                 print('computing %s score...'%(scorer.method()))
---> 62             score, scores = scorer.compute_score(gts, res)
     63             if type(method) == list:
     64                 for sc, scs, m in zip(score, scores, method):

paniquex, [06.03.20 20:58]
~/Documents/Automated_Audio_Captioning_DCASE2020/dcase-2020-baseline/coco_caption/pycocoevalcap/spice/spice.py in compute_score(self, gts, res)
     73         ]
     74         subprocess.check_call(spice_cmd, 
---> 75             cwd=os.path.dirname(os.path.abspath(file)))
     76 
     77         # Read and process results

~/anaconda3/lib/python3.7/subprocess.py in check_call(*popenargs, **kwargs)
    361         if cmd is None:
    362             cmd = popenargs[0]
--> 363         raise CalledProcessError(retcode, cmd)
    364     return 0
    365 

CalledProcessError: Command '['java', '-jar', '-Xmx8G', 'spice-1.0.jar', '/home/paniquex/Documents/Automated_Audio_Captioning_DCASE2020/dcase-2020-baseline/coco_caption/pycocoevalcap/spice/tmp/tmpi2453jj0', '-cache', '/home/paniquex/Documents/Automated_Audio_Captioning_DCASE2020/dcase-2020-baseline/coco_caption/pycocoevalcap/spice/cache', '-out', '/home/paniquex/Documents/Automated_Audio_Captioning_DCASE2020/dcase-2020-baseline/coco_caption/pycocoevalcap/spice/tmp/tmpq7uq3kif', '-subset', '-silent']' returned non-zero exit status 1.

[dr-costas]

Hi,

Thank you for this issue :)

Let me check a bit on that and I will come back.

[dr-costas]

Just to make sure, have you run the script get_stanford_models.sh in the coco_caption directory?

[paniquex]

Yes, I've done it :)

[dr-costas]

Hi,

Now it should be fixed.

[paniquex]

Hi,

Thank you a lot! Everything works well now :)

[Robertwyq]

Hi,

Now it should be fixed.

how to fix this problem?

[dr-costas]

Hi, Now it should be fixed.

how to fix this problem?

Hi,

The above problem is about the binary files that are required for the calculation of the scores. I think that at the moment these files are included in the repo. If not, please try and run the script get_stanford_models.sh in the coco_caption directory.

If the problem still persists, please let me know so we can work it out.