Export data in Nimn Format

[amitguptagwl]

Imglab currently save data in json format. The size of project file can be reduced by 66% if it uses निम्न (nimn) format.

[lopezjorge1]

Hi Amit! I'm new to the open source community and this seems like a doable task. Do you mind if I take a stab at it? Are there any other resources (other than the nimn github) that I could use for reference?

[amitguptagwl]

Welcom @lopezjorge1 . I appreciate your initiative. You're required to change following code in uiaction.js

$("#exportBtn").click(function(){
    download(JSON.stringify(images),"labelled.json","text/plain");
})

Nimn encoder two thing to transform JS object into nimn data: data, schema. To make it quick;

open imglab > import some images > create some boxes > create some points > open console > type JSON.stringify(images)

You'll see the value of object which has to be transformed to nimn data. Now open JSON to Nimn converter. Copy the data into json box and hit convert button. You'll see that it automatically generates schema for you. Use that schema in the application.

Just check that you should not get different schema when you use face++ api. If you get different schema then we'll have to change the structure of images object.

It's a bit tricky. But doable. Let me know if you face any issue.

[lopezjorge1]

Schema without Face++

{
    "lethanos.jpg": {
        "boxes": {
            "1": {
                "left": "number",
                "top": "number",
                "width": "number",
                "height": "number",
                "points": {
                    "1": {
                        "label": "string",
                        "x": "number",
                        "y": "number"
                    },
                    "2": {
                        "label": "string",
                        "x": "number",
                        "y": "number"
                    }...
}

Schema with Face++

{
    "lethanos.jpg": {
        "boxes": {
            "01": {
                "left": "number",
                "top": "number",
                "width": "number",
                "height": "number",
                "attributes": {
                    "gender": {
                        "value": "string"
                    },
                    "age": {
                        "value": "number"
                    },
                    "headpose": {
                        "yaw_angle": "number",
                        "pitch_angle": "number",
                        "roll_angle": "number"
                    }
                },
                "points": {
                    "mouth_upper_lip_left_contour2": {
                        "x": "number",
                        "y": "number"
                    },
                    "mouth_upper_lip_top": {
                        "x": "number",
                        "y": "number"
                    },
                    "mouth_upper_lip_left_contour1": {
                        "x": "number",
                        "y": "number"
                    }, ...
}

These are the two schemas for one image I used in imglab. The schema using Face++ is a lot more specific (attributes "key name" and the points are labeled). This is pretty much the story for every other image as well. Would the structure of the images object have to change now? If not, what step should I take after getting the general structure of the schemas?

[amitguptagwl]

We should come up with the common schema. Eg (roughly)

[{
        "imagename": "lethanos.jpg",
        "boxes": [{
                "left": "number",
                "top": "number",
                "width": "number",
                "height": "number",
                "attributes": [{
                        "label" : "string",
                        "value": ["string"]
                 },
                "points": [{
                        "x": "number",
                        "y": "number",
                        "label" "string"
                  }]
      }]
}]

[lopezjorge1]

Before I continue, the goal is to take the common schema and convert it to the nimn format, then take that data and download it when the export button is clicked, right?

[amitguptagwl]

Goal - Save data in Nimn format instead of JSON format. Precaution : Existing users of the application should not feel interruption.

Tasks

1. Find the common schema so that js object built from different resources can be converted in to common structure. (Nimn supports fixed object structure only)

   {
   "v" : "2.0",
   data : [{
       "imagename": "lethanos.jpg",
       "boxes": [{
               "left": "number",
               "top": "number",
               "width": "number",
               "height": "number",
               "attributes": [{
                       "label" : "string",
                       "value": ["string"]
                },
               "points": [{
                       "x": "number",
                       "y": "number",
                       "label" "string"
                 }]
     }]
   }]
   }

2. Do the necessary changes in the application so that it can use the new changes.

3. After above change, application will not be able to import previously saved data. Hence modify import("Browse" button) functionality which can detect if the data was saved with previous version. And convert old file into new.

Let's make above changes live by raising the PR. After this;

1. Save js object as Nimn data instead of json data and download when user clicks on "Save" button
2. In case of import ("Browse" button), if the file is json use JSON.parse(filedata) or if it is nimn then use nimn.decoder(filedata)

[amitguptagwl]

Done. Schema: js/nimnObjStructure.js

//save file
download( nimn.stringify(nimnSchema, labellingData), fileName, "application/nimn");

//open file
labellingData = nimn.parse(nimnSchema, data);