JSON editor and validator for metadata generator

[fedorov]

It might be helpful to have the component showing the output of the metadata JSON generator editable, and also add a button to do very basic JSON validation (not schema-based, for now, but at least check correctness of JSON).

@che85 what do you think? do you think it would be useful?

[che85]

What do you mean with "output of the metadata JSON generator editable"?

I agree with the JSON validator, which actually should be done automatically as soon as the user requests to generate it. So I would leave it the way it is but add the basic validation and add some notification bar (or something similar)

[fedorov]

I was thinking that in case user wants to change something in the JSON, it is better to allow that directly in the browser, and do validation after the change, rather than having user copy it to some other editor and change it there.

[che85]

What would you expect the user to change? Everything the user should be able to edit, can be added through the web components.

Otherwise for instance the user could change the segment number to an already existing one and the basic json validation would not even notice that.

If the user wants to edit the output, he/she should copy it to another editor. Then it's at least not our fault if something goes wrong ;)

[fedorov]

It will be difficult to always keep web app in sync with the schema. It may be overwhelming to expose all options of the schema in the webapp interface.

Anyway, let me finish the specific use case I am working on first.

[fedorov]

As a specific example, I changed capitalization to segmentAttributes, and now the output is invalid. Took me a little bit to debug that segfault...

[fedorov]

Another one - algorithm name/type missing in webapp output

[che85]

Ok please finish the schema definition and I will add all attributes that you want me to add there.

Another thing would be generating the web ui based on the schema... but that's something for the future I guess and I don't know how trivial that is.

[fedorov]

that is exactly one of the points - if it cannot be automatically updated from the schema, it will be hard

and the second point - if it IS updated from the schema, it may be too much to expose in the web app

agree that we can decide on this in the future

[che85]

@fedorov This validator seems to be pretty good: http://www.jsonschemavalidator.net/

[che85]

It also gives me the hint, that there is something missing in https://github.com/QIICR/dcmqi/blob/master/doc/pm-schema.json#L68

[fedorov]

It also gives me the hint, that there is something missing in https://github.com/QIICR/dcmqi/blob/master/doc/pm-schema.json#L68

right, but that is just not a valid JSON, so it will fail any json validator. Are there any validators that could allow specifying external schema?

[che85]

You are right! I will need to check that. Will get back to you later with some results

[che85]

Copy, pasting the json schema works, but it seems not to be valid the way it is defined.

We should also take care of the correct links in those schema files since http://qiicr.org/schema/dcmqi/common-schema.json does not exists yet.

[che85]

I am going to look into the issues...

[che85]

Please try http://www.jsonschemavalidator.net/

Schema:

{
  "@context": "http://qiicr.org/dcmqi/contexts/dcmqi.jsonld",
  "id": "http://qiicr.org/schema/dcmqi/pm-schema.json",
  "$schema": "http://json-schema.org/draft-04/schema#",

  "required" : ["DerivedPixelContrast", "AnatomicRegionCode", "FrameLaterality"],
  "properties": {
    "SeriesDescription": {
      "allOf": [
        {
          "$ref": "#/definitions/LO"
        },
        {
          "default": "Segmentation"
        }
      ]
    },
    "SeriesNumber": {
      "allOf": [
        {
          "$ref": "#/definitions/IS"
        },
        {
            "default": "300"
        }
      ]
    },
    "InstanceNumber": {
      "allOf": [
        {
          "$ref": "#/definitions/IS"
        },
        {
            "default": "1"
        }
      ]
    },
    "BodyPartExamined": {
      "allOf": [
        {
          "$ref": "#/definitions/CS"
        },
        {
            "default": ""
        }
      ]
    },
    "QuantityValueCode": {
      "$ref": "#/definitions/codeSequence",
    },
    "MeasurementUnitsCode": {
      "$ref": "#/definitions/codeSequence",
    },
    "MeasurementMethodCode": {
      "$ref": "#/definitions/codeSequence",
    },
    "RealWorldValueSlope": {
      "$ref": "#/definitions/FD",
      "default": "1.0"
    },
    "DerivedPixelContrast": {
      "enum": [
        "ADDITION",
        "DIVISION",
        "MASKED",
        "MAXIMUM",
        "MEAN",
        "MINIMUM",
        "MTT",
        "MULTIPLICATION",
        "RCBF",
        "RCBV",
        "RESAMPLED",
        "STD_DEVIATION",
        "SUBTRACTION",
        "T_TEST",
        "TTP",
        "Z_SCORE",
        "MIXED",
        "ADC",
        "DIFFUSION",
        "DIFFUSION_ANISO",
        "DIFFUSION_ATTND",
        "DIFFUSION_ISO",
        "FAT",
        "FAT_FRACTION",
        "FIELD_MAP",
        "IN_PHASE",
        "METABOLITE_MAP",
        "NEI",
        "OUT_OF_PHASE",
        "PERFUSION_ASL",
        "R_COEFFICIENT",
        "R2_MAP",
        "R2_STAR_MAP",
        "RHO",
        "SCM",
        "SNR_MAP",
        "T1_MAP",
        "T2_STAR_MAP",
        "T2_MAP",
        "TCS",
        "TEMPERATURE",
        "VELOCITY",
        "WATER",
        "WATER_FRACTION"
      ]
    },
    "AnatomicRegionCode": {
      "$ref": "#/definitions/codeSequence",
    },
    "FrameLaterality": {
      "$ref": "#/definitions/CS",
      "default": "U",
    }
  },
  "definitions": {
    "PN": {
      "type": "string",
      "maxLength": 64
    },
    "CS": {
      "type": "string",
      "maxLength": 16
    },
    "IS": {
      "type": "string",
      "maxLength": 12
    },
    "LO": {
      "type": "string",
      "maxLength": 64
    },
    "FD": {
      "type": "number"
    },
    "SH": {
      "type": "string",
      "maxLength": 64
    },
    "codeSequence": {
      "type": "object",
      "required": [
        "CodeValue",
        "CodingSchemeDesignator",
        "CodeMeaning"
      ],
      "properties": {
        "CodeValue": {
          "allOf": [
            {
              "$ref": "#/definitions/SH"
            },
            {
                "default": "T-D0050"
            }
          ]
        },
        "CodingSchemeDesignator": {
          "allOf": [
            {
              "$ref": "#/definitions/SH"
            },
            {
                "default": "SRT"
            }
          ]
        },
        "CodeMeaning": {
          "allOf": [
            {
              "$ref": "#/definitions/LO"
            },
            {
                "default": "Tissue"
            }
          ]
        }
      }
    }
  }
}

and data:

{
  "SeriesDescription": "Apparent Diffusion Coefficient",
  "SeriesNumber": "701",
  "InstanceNumber": "1",
  "BodyPartExamined" : "PROSTATE",
  "QuantityValueCode": {
    "CodeValue": "113041",
    "CodingSchemeDesignator": "DCM",
    "CodeMeaning": "Apparent Diffusion Coefficient"
  },
  "MeasurementUnitsCode": {
    "CodeValue": "mm2/s",
    "CodingSchemeDesignator": "UCUM",
    "CodeMeaning": "square millimeter per second"
  },
  "AnatomicRegionCode": {
    "CodeValue": "T-9200B",
    "CodingSchemeDesignator": "SRT",
    "CodeMeaning": "Prostate"
  },
  "FrameLaterality" : "U",
  "RealWorldValueSlope" : 1e-6,
  "DerivedPixelContrast" : "ADC"
}

[che85]

Right now for checking I added the definitions into the same json to prevent errors with referencing to a non existing url.

Also type FD was not defined.

[che85]

https://json-schema-validator.herokuapp.com/ seems also to be pretty nice since it also takes care of unexpected "," in the end of a line where it would not be needed. (it's just a matter of taste I think)

[che85]

I also got the feeling, that default values are never used as fallback values by json validators. So when an attribute is required and a default value is specified, it will still fail

[che85]

I am also adding a restriction to json for RGB values since that should not be of DICOM VR US. Only in range of 0 to 255

[che85]

What is the range of CIELab values? Is that 0 to 2E+16?

[fedorov]

There are many constraints that we can add, but I am interested in whether there are validators that can check the structure of the document against the schema, without adding all of the type constraints. Does this work if you, for example, manually specify the schema URLs pointing to github? (for those validators you looked at)

I don't know what are the constraints for CIELab values.

[che85]

Let me check. I think it should work.

[che85]

This page lists some validators http://json-schema.org/implementations.html, but it seems that there is only that famous one, that I already posted for online validation

[che85]

I think it should be possible to use our webapp for that. I could use an api for validating against our schema and you would just past your code into the textarea on the right side.