Closed vikas-agarwal76 closed 2 years ago
ohsh6o
commented
3 years ago @vikas-agarwal76 I saw this and just read through it but will take me some time to break down a response point by point. Thank you for the feedback, it is very valuable! I will try to respond more completely later today!
ohsh6o
commented
3 years ago We are doing an initial investigation to enable tooling for automatic validation of OSCAL documents for fedramp compliance based on the extension specification in "resources/json/FedRAMP_extensions.json" file . We have some clarifications on the intent and usage of some of the extension constructs, some issues that we identified, and also some suggestions. These are captured below.
Excellent, I am happy to have you here!
1. Some of the required fields mentioned in pdf files - both common across different OSCAL models, and model specific things are not captured in FedRAMP_extensions.json. For example - 1. “fedramp-pmo” is a required role 2. FedRAMP logo should be included in back-matter/resources and linked in Party representing fedramp-pmo 3. There should be a responsible-party with role-id as “fedramp-pmo” associating it to FedRAMP party. 4. “system-owner” role is required in SSP. 5. etc.
Some of those are likely bugs then. I think we will want to improve and enhance the FedRAMP extensions file format.
2. Is there some documentation which describes the meaning of various fields used in FedRAMP_extensions.json? What can be the expected structure of this file and details of each field / sub-field, possible options? Whether a field is required or not, etc. so that we can process this file programmatically.
Generally speaking, no. OSCAL 1.0.0 (and not its Metaschema source code) does not have an optimal way to show these constraint items (allowed values, minimal and maximal occurences, required or optional status, data type, et cetera) and so both NIST OSCAL and FedRAMP extensions are different attempts of explaining this data out of band.
3. What is the purpose of “indexes”?
The purpose of an index is to say, for a given field, its value you should or must refer to another field in the document model. For example, if you have an element that has an index to a back-matter resource's @uuid, you are saying that field (depending on how it is defined) either should or must be a UUID reference to an existing resource with that UUID or it is invalid.
As your questions suggest, it is easier to write XPath and Schematron to express that. A couple of community implementations around doing so in JSON exist, but none are standard.
4. Does "extension-namespace" field means all properties captured in “extensions” belong to this namespace?
Yes, correct.
5. There are 2 “extensions” field - top level, and actual extensions. They should be named differently.
There is much room for improvement in the previous "values" file (the validations development team relies on this more heavily, FYI/FYE) and the extensions file, but yes, that can be taken into consideration. The idea is much like that of a "components" file, which had a top-level definition component-definitions with an array of components. At runtime, serializing these into XML and JSON makes it easier to merge a series of objects.
I can put a recommendation in the backlog to consider renaming then? There are many additional considerations I would like to consider with enhancing the extensions format.
6. Does "extension-name" under extensions give the name of the “prop” in “bindings”? If yes, the at some places the name pf the prop is explicitly captured in bindings and sometimes it is same as "extension-name" and sometimes different.
They can be different. The name would allow you to given it a free-form name, the binding tells you which specific OSCAL elements can be extended. We in FedRAMP have just not been giving creative or divergent names yet.
7. The "revision-history-party-uuid" extension has min. Cardinality of 0, whereas as per pdf it is required.
Thank you. I will file a bug in regards to this.
8. The usage of different fields in “index-has-keys" (for revision-history-party-uuid) is not very clear, even though the intent is clear.
OK, can you tell me what would make it clearer for you? Any recommendations?
9. The “pattern” specified in bindings is not consistent; sometimes it is full path, sometimes it is relative, sometimes it includes name, and other attributes also. What are different variations possible and how to interpret them? For example - in “authenticator-assurance-level'"bindings": [ { "pattern": "/o:system-security-plan/o:system-characteristics/o:prop" }, { "pattern": "system-characteristics/o:prop[@name='authenticator-assurance-level']" }, { "pattern": "system-characteristics/o:prop[@name='federation-assurance-level']" }, { "pattern": "system-characteristics/o:prop[@name='identity-assurance-level']" } ],
This is due to the inherent nature of XPath. If you were to load all object models at runtime (SSP, SAP, SAR and POA&M), with XPath you can say, given a particular context, I need to see specifically one or more props in the system-characteristics of a system-security-plan ("pattern": "/o:system-security-plan/o:system-characteristics/o:prop") or find one or more props in the system-characteristics in any child from the root (not just system-security-plan). This is inherent to XPath querying, so we make some queries more loose where it is more practical than specifying each, specific pathing query with the full path for n>1 paths. Some of those in some cases I might not be able to fully predict. XPath is flexible that way.
In "interconnection-security”. What is the purpose of @value
"bindings": [ { "pattern": "o:component[@component-type='interconnection']/o:prop[@name='connection-security'][@ns='https://fedramp.gov/ns/oscal']/@value" } ]
As this is a prop, this is saying you will match specifically on its value attribute in the XML form of this field. Let's consider this example.
<prop name="connection-security" ns="https://fedramp.gov/ns/oscal" value="ipsec">
<remarks>
<p>If "other", remarks are required. Optional otherwise.</p>
</remarks>
</prop>The query o:component[@component-type='interconnection']/o:prop[@name='connection-security'][@ns='https://fedramp.gov/ns/oscal']/@value will return only the right hand side data in @value, ipsec, whereas o:component[@component-type='interconnection']/o:prop[@name='connection-security'][@ns='https://fedramp.gov/ns/oscal'] (sans @value) should return all of the matching object, so what's below.
<prop name="connection-security" ns="https://fedramp.gov/ns/oscal" value="ipsec">
<remarks>
<p>If "other", remarks are required. Optional otherwise.</p>
</remarks>
</prop>In "transport-type" "pattern": "o:component[@component-type='service']/o:protocol/o:port-range/@transport"
In "no-oscal-ssp-title-short" "pattern": "/o:assessment-plan/o:back-matter/o:resource[./o:prop[@name='type'][.='no-oscal-ssp']]/o:prop"
1. How to interpret “expects” field - example - in "sorn-id""expects": [ { "test": ".[@name='pta-4'][@ns='https://fedramp.gov/ns/oscal']/@value='yes' and .[@name='sorn-id'][@ns='https://fedramp.gov/ns/oscal']" } ]
If you answer yes to the Privacy Threshhold Analysis questionnaire and you respond to question 4/4 with an answer of yes, your system should also have a System of Record Notice Identifier registered for that information system's SSP. I guess we would call that conditional branching logic?
In "planned-completion-date" constraint
"expects": [ { "test": "(o:prop[@name='implementation-status'][@ns='https://fedramp.gov/ns/oscal'][@value='partial']/remarks)" }, { "test": "(o:prop[@name='implementation-status'][@ns='https://fedramp.gov/ns/oscal'][@value='planned']/remarks)" }, { "test": "(o:prop[@name='implementation-status'][@ns='https://fedramp.gov/ns/oscal'][@value='alternative']/remarks)" }, { "test": "(o:prop[@name='implementation-status'][@ns='https://fedramp.gov/ns/oscal'][@value='not-applicable']/remarks)" } ]
If you do not answer with an implementation-status with a @value of implemented and rather use any of these for a control implementation statement, you must explain yourself for each one of those cases with a remarks field, or this is an incomplete statement that will get rejected.
In “Port Class Exists” constraint "test": "exists(@Class)"
Given the example below, the @class attribute (in XML form) is mandatory and must exist. We are not checking value, just its existence.
<prop name="port" class="remote" value="80" ns="https://fedramp.gov/ns/oscal"/>1. In many cases "allowed-values" does not have "allow-other”? How should this be interpreted - “yes” or “no”?
The default that is implied "allow-other”="yes" and you can have unconstrained values. When you see "no" these are effectively the only values FedRAMP will accept and our validation tool will flag other values as concerns, i.e. your package is deficient.
2. What is the difference between extensions and constraints? Constraints also define new properties. For example - in "sar-risk-impacted-control", binding "pattern": "/o:assessment-results/o:result/o:risk/o:prop"
Extensions can add new fields (often props but not always). Extensions can have constraints. Constraints are limitations that can apply to existing fields (but in NIST OSCAL's Metaschema source code these are obscured but the constraints exist in OSCAL's definitions). So you can add new fields with extensions, or use extensions to customize and limit how existing fields are used (by constraining existing fields, not adding new ones).
Does this summary help clarify?
3. Why is extensions an object with each extension represented as “id”: {}, whereas, constraint is an array [].
Is that the case? It seems constraints are object references.
"id": "users-external-future",
"extension-name": "users-external-future",
"formal-name": "Future External Users",
"description": "The anticipated number of users external to the organization in one year.",
"binding": {"pattern": "/o:system-security-plan/o:system-characteristics/o:prop"},
"constraint": {
"matches": {"data-type": "integer"},
"has-cardinality": {
"min-occurs": 1,
"max-occurs": 1
}
}
},4. In constraint - "planned-completion-date", there is property field. What does this means?
This seems to no longer exist.
"props": [ { "name": "reference", "value": "3.1" } ],1. If caridnality is not present then how it is to be interpreted? For example - in "attachment-type" constraint.
Generally, speaking, the default is a min and max of 0 to unbounded (infinite) unless otherwise stated. This is explained elsewhere in the OSCAL docs.
Here are some suggestions -
1. Use JSONPath for extensions specification in JSON format rather than XPath which is applicable for XML. 2. Bindings should be broken in 2 parts - Context - set of elements/fields on which this will apply e.g., /*/o:metadata/o:revisions/o:revision Binding - actual element(s) on which the constraints have to be evaluated for each element of the context - o:prop[@name=party-uuid] 3. Constraint should have an additional field “target” on which the constraint would apply - “target”: @value. Also the constraint should be an array and grouped accordingly. Some constraints apply on value. Others apply on other attributes of binding, like @Class attribute should exists or should have a specific value, etc.
These are excellent recommendations and would like to explore further in a future revision. Like I said, there are many considerations on improving, normalizing, and standardizing an extension format in XML and JSON for more general use by the NIST OSCAL ecosystem, not just FedRAMP. Shall I make an issue to track these requests?
ohsh6o
commented
3 years ago @vikas-agarwal76, did you have a chance to review the extensions and values data after our touch base earlier this week with @chrisbutler54 and @fsuits. Was this helpful or do we need to set aside more time to discuss further?
FYI, I will be out next week on leave, but feel free to post responses or new inquiries and I will get to them as soon as I can!
ohsh6o
commented
3 years ago How is it going, @vikas-agarwal76? Were you able to use the updated guidance and code from our 10x team and NIST OSCAL devs to move forward?
ohsh6o
commented
2 years ago Hi @vikas-agarwal76, we have not heard from you in a while. I will close for now. Feel free to re-open or, better yet, open a specific issue with feature requests or bug reports as they arise. Thanks again for reaching out!
vikas-agarwal76
commented
2 years ago @ohsh6o Hi Alexander, The code to validate FedRAMP including conversion from JSON to XML is working. We are currently exploring ways to integrate this into the main trestle project. The main issue here is because SaxonC cannot be installed using pip, hence it introduces lots of complexities in our github CI process ( the trestle CI process is designed to run on linux, mac, and Windows). The installation and setup of SxaonC needs to be done for each platform for our github workflows to run properly. We are exploring ways to see how this part of the code can be added as plugin to trestle so that rest of the things work smoothly if SaxonC is not installed. Its fine to close this issue as we got the FedRAMP validation working. If we face any problem then will open a specific issue for that.
We are doing an initial investigation to enable tooling for automatic validation of OSCAL documents for fedramp compliance based on the extension specification in "resources/json/FedRAMP_extensions.json" file . We have some clarifications on the intent and usage of some of the extension constructs, some issues that we identified, and also some suggestions. These are captured below.
In "interconnection-security”. What is the purpose of @value
In "transport-type" "pattern": "o:component[@component-type='service']/o:protocol/o:port-range/@transport"
In "no-oscal-ssp-title-short" "pattern": "/o:assessment-plan/o:back-matter/o:resource[./o:prop[@name='type'][.='no-oscal-ssp']]/o:prop"
How to interpret “expects” field - example - in "sorn-id"
In "planned-completion-date" constraint
In “Port Class Exists” constraint "test": "exists(@class)"
In many cases "allowed-values" does not have "allow-other”? How should this be interpreted - “yes” or “no”?
What is the difference between extensions and constraints? Constraints also define new properties. For example - in "sar-risk-impacted-control", binding "pattern": "/o:assessment-results/o:result/o:risk/o:prop"
Why is extensions an object with each extension represented as “id”: {}, whereas, constraint is an array [].
In constraint - "planned-completion-date", there is property field. What does this means?
If caridnality is not present then how it is to be interpreted? For example - in "attachment-type" constraint.
Here are some suggestions -
Use JSONPath for extensions specification in JSON format rather than XPath which is applicable for XML.
Bindings should be broken in 2 parts - Context - set of elements/fields on which this will apply e.g., /*/o:metadata/o:revisions/o:revision Binding - actual element(s) on which the constraints have to be evaluated for each element of the context - o:prop[@name=party-uuid]
Constraint should have an additional field “target” on which the constraint would apply - “target”: @value. Also the constraint should be an array and grouped accordingly. Some constraints apply on value. Others apply on other attributes of binding, like @Class attribute should exists or should have a specific value, etc.