Error using macvi-usv-odce-tool evaluate code

[NeelBhowmik]

Hi, Thanks for providing the evaluation tools. However, I am getting an error while running macvi-usv-odce-tool evaluate:

macvi_usv_odce_toolkit/evaluation.py", line 65, in convert_to_coco_structures dataset = dataset['dataset'] KeyError: 'dataset'

I am trying to run on val set, and the inputs are: macvi-usv-odce-tool evaluate <path to dataset>/lars_v1.0.0_annotations/val/panoptic_annotations.json <path to result>/results.json

dataset-json-file: lars_v1.0.0_annotations/val/panoptic_annotations.json
results-json-file: results.json

The results.json is created according to the instructions.

{
  "info": {},
  "images": [
    {
      "id": 3995,
      "width": 1280,
      "height": 720,
      "file_name": "yt028_01_00030.jpg"
    },
    <...>
  ],
  "annotations": [
    {
      "image_id": 3995,
      "file_name": "yt028_01_00030.png",
      "detections": [
        {
          "id": 1,
          "bbox": [
            0,
            0,
            1280,
            413
          ],
        },
        <...>
      ]
    },
    <...>
  ]
}

Could you please suggest what could be the possible reasons for this error?

Thanks.

[rokm]

Looks like the dataset JSON is missing the dataset key. Given the indicated path to the dataset, my guess would be that you are trying to run the evaluation tool against an unsupported dataset?

This evaluation kit was intended for MODS dataset (see for example mods.json in mods-lite.zip).

[NeelBhowmik]

I am trying to use this code for LaRS dataset onval set (USV-based Obstacle Detection challenge). LaRS comes with. Val set comes with panoptic_annotations.json and image_annotations.json files; and I have generated results.json (the same format as suggested on the instruction page). I am following the instructions on this GitHub page (attached image). So, to evaluate the LaRS dataset, is this not the correct code or am I missing something?

Thanks,

[rokm]

So, to evaluate the LaRS dataset, is this not the correct code or am I missing something?

This evaluation kit was written specifically for MODS dataset dataset, so it likely won't work with other datasets, such as LaRS.

The README in this repository does not mention LaRS anywhere - I have no idea where you copied that modified Section 4. Evaluate the results from...

[NeelBhowmik]

Thanks for the clarification.