Support to validate different / multiple conformance objectives.

[butler54]

Issue description / feature objectives

Fedramp (as an example) have defined expected values / conformance within the OSCAL schema (see here:https://github.com/GSA/fedramp-automation/blob/master/documents/FedRAMP_OSCAL_Registry.xlsx) It would be a good feature if validation could be extended with a conformance format of some type.

Completion Criteria

[butler54]

Will add more details - a placeholder for now.

[butler54]

Based on discussion with Anca this is a need. The question is what language with which we express constraints. This looks promising https://github.com/amer-ali/jsontron - much like schematron would look like for xml.

[butler54]

This issue is on validate an acceptable design only. I've tagged it for Sprint 2 with Shaila and Jeff both on this. I think this may be more work than we think.

[shaila309]

@jeffdmgit Here is the jsontron tutorial as discussed: https://amer-ali.github.io/jsontron/Jsontron-tutorial-v1.pdf

[shaila309]

Related issues open in usnistgov/metaschema

• Tracker Issue for Metaschema work: https://github.com/usnistgov/OSCAL/issues/633
• Integrating XSLT-M4 Metaschema into OSCAL repo: https://github.com/usnistgov/OSCAL/issues/721

Summary (need-to-be-verified)

• Extra metadata constraint validation is done on XML using schematron.
• To validate extra metadata constraint on JSON, first the JSON is converted to equivalent XML and then schematron is applied on that.

Code

The integration branch https://github.com/david-waltermire-nist/OSCAL/tree/metaschema-m4-integration has a working version of this.

[shaila309]

Data validation using jsontron https://github.com/amer-ali/jsontron - Example

• Goal: All assessment result observation properties MUST be of ns (namespace) "IBM"

• Sample JSON to validate sample-OSCAL-assessment-result-observation.json

  {
  "sample_ar_observation":{
  "uuid":"",
  "title":"CIS CentOS Linux 7 Benchmark 2.2.0:1.1.1.1, 192.168.122.4, Sep 11, 2020 at 5:12 PM AEST",
  "description":"IBM S&CC scan, CIS CentOS Linux 7 Benchmark 2.2.0:1.1.1.1, 192.168.122.4, 2020-09-11T17:12:00+1000",
  "properties":[
      {
          "ns": "IBM",
          "class": "id",
          "name": "scan",
          "value": "SGA Centos validation - Sep 11, 2020 at 5_12 PM AEST"
      },
      {
          "ns": "IBM",
          "class": "id",
          "name": "profile-name",
          "value": "CIS CentOS Linux 7 Benchmark 2.2.0"
      },
      {
          "ns": "IBM",
          "class": "id",
          "name": "rule",
          "value": "1.1.1.1"
      }
  ]
  }
  }

• Sample rule to be used by jsontron rule-OSCAL-assessment-result-observation.json

  {
  "schema": {
  "id": "OSCAL AR Observation Rules",
  "title": "Schematron Semantic Validation",
  "schemaVersion": "V1",
  "queryBinding": "jsonpath",
  "pattern": [
    {
      "id": "observation_properties",
      "title": "OSCAL AR Observation Properties",
      "abstract": false,
      "rule": [
        {
          "id": "ns_ibm",
          "abstract": false,
          "context": "$.sample_ar_observation.properties.*",
          "assert": [
            {
              "id": "assertid1",
              "test": "jp.query(contextNode, '$..ns') == 'IBM'",
              "message": "Assert 1: All observation properties MUST be of ns (namespace) IBM"
            }
          ]
        }
      ]
    }
  ]
  }
  }

• Command node $JSONValidator -i sample-OSCAL-assessment-result-observation.json -r rule-OSCAL-assessment-result-observation.json -d

• Output

Starting Semantic Validation ......... Parsing Pattern: observation_properties 1 Pattern(s) Requested. 1 Pattern(s) Processed. 0 Pattern(s) Ignored. THIS INSTANCE IS SEMANTICALLY VALID Completed Semantic Validation ......... Total Errors Found: 0 Total Warnings Found: 1 Total Validations: 3 Total Failed Assertions: 0 Full Validation Report :

Report {
  errors: [],
  warnings: [
    {
      schInstance: [Object [global]],
      schema: [Object],
      attribute: 'Phase',
      message: 'Parsing Warning. No phase found ',
      detail: "This schema doesn't have any phase defined . All Patterns will be processed."
    }
  ],
  validations: [
    {
      schRule: [Object],
      ruleContext: [Array],
      assertionid: 'assertid1',
      assertionTest: "jp.query(contextNode, '$..ns') == 'IBM'",
      message: 'successful',
      assertionValid: true
    },
    {
      schRule: [Object],
      ruleContext: [Array],
      assertionid: 'assertid1',
      assertionTest: "jp.query(contextNode, '$..ns') == 'IBM'",
      message: 'successful',
      assertionValid: true
    },
    {
      schRule: [Object],
      ruleContext: [Array],
      assertionid: 'assertid1',
      assertionTest: "jp.query(contextNode, '$..ns') == 'IBM'",
      message: 'successful',
      assertionValid: true
    }
  ],
  finalValidationReport: [],
  valid: true
}

[shaila309]

Oct 21: Discussion

• Integrate with external tool like jsontron (written in javascript) is not a preferred option as it will have lots of dependency.
• Develop a mini validator in Python by ourselves like jsontron could be an option, but it will create many additional models on top of basic oscal models and will require lots of parsing.
• Preferred option is to define a factory style class (abstract class interface in Python) which will receive the object to do validation.

[butler54]

Attachched image based on latest discussion.

[shaila309]

Oct 26: Discussion

• There can be different types of conformance validators for different parties like ibm, fedramp, etc. as well as a default cross structural conformance validator.
• The expected order of operation: basic oscal schema validation, cross id (uuid) uniqueness validation, default cross structural conformance validator, selected conformance validator (ibm, fedramp, etc.)
• The overlay of validators on top of basic socal schema validator can be optional.
• Validation can happen both Inter document and intra document.

From validation perspective, the expected scenario is:

• Input: one or more oscal object
• Output: pass / fail
• Log: the reason behind the output

To be discussed:

• How do we mention which validator we want to use if there are many?
• What will be the interface to specify the constraints?

[shaila309]

@butler54 @jeffdmgit Here is the response from the OSCAL team on the issue of Validating extra-schema constraints over JSON: https://github.com/usnistgov/OSCAL/issues/726

[butler54]

@vikas-agarwal76 - based on the exchange protocol requirements we should revisit this as an issue.