UniversalDataTool / udt-format

A simple universal data description format for datasets, tailored for interfacing with humans.
https://universaldatatool.com
MIT License
20 stars 14 forks source link

Support for linked annotation nesting #3

Open Ownmarc opened 4 years ago

Ownmarc commented 4 years ago

We are often training multiple models that search for specific things in an image. When found, they are passed to other models of their corresponding category.

For example : First model's job is to find the "big" objects and then pass it to the right second model specificly trainned for this big image class to find the "smaller-parts". This is a 2 level nesting, but we should be able to add more!

I didn't find an annotation format / tool that supports that yet. Also, a change to "parent" object should be changing (or not) the children's position following the user defined strategy. For example, resizing "left" side of a bounding box for -15px should translate all the children positions for -15px if the nested change strategy is telling to do so.

Here is a basic example of nested annotations :

{
  "filename": "12345.jpg",
  "key": "12345",
  "size": {
    "width": 4608,
    "height": 2240,
    "depth": 3
  },
  "annotations": {
    "ground_truth": {
      "objects": [
        {
          "name": "building",
          "bndbox": {
            "left": 600,
            "top": 100,
            "right": 2000,
            "bottom": 1000
          },
          "size": {
            "width": 1400,
            "height": 900,
            "depth": 3
          },
          "annotations": {
            "ground_truth": {
              "objects": [
                {
                  "name": "window",
                  "bndbox": {
                    "left": 100,
                    "top": 34,
                    "right": 853,
                    "bottom": 130
                  }
                },
                {
                  "name": "window",
                  "bndbox": {
                    "left": 100,
                    "top": 34,
                    "right": 853,
                    "bottom": 130
                  }
                },
                {
                  "name": "door",
                  "bndbox": {
                    "left": 100,
                    "top": 134,
                    "right": 430,
                    "bottom": 230
                  }
                },
                {
                  "name": "door",
                  "bndbox": {
                    "left": 100,
                    "top": 134,
                    "right": 430,
                    "bottom": 230
                  }
                }
              ]
            }
          }
        }   
      ]
    }
  }
}
seveibar commented 4 years ago

Multiple stages of annotations based on the results of previous stages. Some type of conditional logic within a composite interface perhaps.