search

usnistgov / OSCAL

Open Security Controls Assessment Language (OSCAL)
https://pages.nist.gov/OSCAL/
Other
674 stars 183 forks source link

Valid OSCAL File Fragments #590

Open brian-ruf opened 4 years ago

brian-ruf commented 4 years ago

User Story:

As an OSCAL content user, I sometimes need to exchange only a portion of an OSCAL file (file fragment) with other parties. For example, the entire SSP must be reviewed annually, but the system inventory within the SSP must be submitted monthly (along with scanning data and POA&M snapshots).

I need a way to send an OSCAL file fragment that still allows OSCAL validation tools to recognize the file fragment as valid.

Goals:

Provide a mechanism for exchanging and validating OSCAL file fragments.

Dependencies:

None.

Acceptance Criteria

brian-ruf commented 4 years ago

Case Study: As mentioned in the issue above, the SSP model includes a system inventory. FedRAMP requires a CSP to deliver the entire SSP (including system inventory) to FedRAMP annually. FedRAMP also requires a CSP to deliver their up-to-date system inventory monthly (along with monthly system scans and POA&M snapshot).

The CSP should be able to extract the relevant data under /system-implementation/system-inventory from the System Security Plan and deliver it. In this example, the file must still have other OSCAL content that is not relevant to system-inventory, such as an import-profile assembly, or it will fail validation.

We need a mechanism that indicates this is an OSCAL file fragment, thus requires relaxed validation.

Minimally, a validation mechanism would still enforce the minimum requirements of any OSCAL file (title, last-modified, UUID, etc.), yet would relax any model-specific cardinality rules to zero. (No field would be "required" except as noted above.)

Ideally, a validation mechanism could be configured to identify which portions of the OSCAL file fragment should be subject to more complete validation requirements. For example, the validation mechanism could be directed to still enforce required fields for children of system-inventory, even though it is relaxing those requirements for the rest of the ssp-specific structure.

bradh commented 4 years ago

This feels like maybe a different format within the implementation layer. Possibly the implementation layer needs to be finer grained, and use some kind of include mechanism.

brian-ruf commented 4 years ago

@bradh I'm open to any solution that allows OSCAL file fragments. This came up verbally a few weeks ago with @david-waltermire-nist and @wendellpiez, but other topics overshadowed our plan to discuss it. For now It's more of a place-holder to make sure we don't lose sight of it.

wendellpiez commented 4 years ago

As for today's question: on the XML side, a document rooted at system-inventory will be valid. How it is to be integrated with other SSP pieces in a workflow is not defined (the other and more difficult part of Brian's question), but formal validity of such a fragment is not a problem.

This is not currently the case on the JSON side, but we know how to ameliorate that. The current thinking is to provide anything defined as an assembly in a metaschema, as a valid root for an instance.

How document assembly -- an include mechanism -- is to be implemented is a somewhat different question. XML has two standard widely adopted mechanisms -- external parsed entities, and XInclude -- offering different affordances and tradeoffs. (For the curious I can be more specific.) Both of these deliver functionality as specified, are well understood, and are implemented in commodity tooling; this makes me reluctant to reinvent that wheel. Especially since on the JSON side things appear more more fluid. (No one seems to have thought much about how to model hypertext resources in JSON. Heh. JSON pointer - as I understand, designed to support graph structures - might be useful.)

All that leads me to feel that managing dynamic assembly of OSCAL instances from fragments is an implementation question, not something that needs to be supported by the standard. (Yet.)

On the other hand, the need stated by @brianrufgsa is real, and we can help him with that.

david-waltermire commented 4 years ago

@wendellpiez and @david-waltermire-nist will organize some thoughts on this topic. A meeting to discuss this issue will follow.

wendellpiez commented 4 years ago

@brianrufgsa could you please restate the requirement here?

Note that currently, an XML document rooted at system-inventory may be schema-valid against the SSP XSD. Assuming all its pieces are correct, etc. (Maybe the documentation should be updated so as not to create an impression this is not the case?)

A JSON document rooted at system-inventory will not today be valid against the SSP JSON Schema; if we make this modification to permit it (given this is already okay using XML/XSD) would that address the need?

Yet the complexity of the choreography here (doc A is submitted annually but its logical or nominal component Ab is submitted monthly) makes me think that schema validation alone is not going to support all applicable constraints (even automatable ones): there will be other considerations as well and other kinds of checking to be performed including content-checking.

So I am also led to wonder (oppositely) if the problem isn't actually deeper: does a standalone system inventory need its own document-level metadata? Maybe this is about not only schema validation, but modeling as well. Adding metadata as an optional element inside system-inventory and perhaps other elements could be a solution ... to some problems. Not yet sure if that would address the use case -- and it does introduce other complications.

brian-ruf commented 4 years ago

@wendellpiez, simply putting system-inventory at the root may be valid syntax, but does not address the requirement.

Any inventory fragment must at least include the system name, system owner's organization (CSP name), and a publication date. These are snapshots in time of the inventory, which are correlated with security scans logs (also snapshots in time), thus the publication date is important for processing.

Further, the system-inventory assembly relies on component assembly, and the metadata assembly (to reflect who is responsible for the components and inventory-items).

To properly reflect the inventory as an SSP fragment using the current SSP syntax, I would need the following assemblies/fields:

Unfortunately, I suspect most fragment examples, are going to be more complex than just using an assembly at the root. This is why I verbally suggested an approach where we can put a directive at the top of the OSCAL file which essentially indicates to syntax validation tools that required fields should not be enforced.

Perhaps using such a directive would still require title, last-modified, and an updated root @id attribute, and still performing all other syntax checks. Essentially this would treat all other fields/assemblies with a cardinality of "1" as if they were "0 or 1", and all fields/assemblies with a cardinality of "1 or more" as if they were "0 or more". All other syntax validation would process normally.

I would love to see something like a fragment attribute at the root element, so I could specify something like:

<system-security-plan id="12345" fragment="inventory">
   <metadata>
      <!-- detail -->
   </metadata>
   <system-characteristics>
      <system-name>This System's Name</system-name>
   </system-characteristics>
   <system-implementation>
      <component>
         <!-- detail -->
      </component>
      <system-inventory>
         <!-- detail -->
      </system-inventory>
   </system-implementation>
</system-security-plan>

This would allow syntax validation tools to still process the file, relaxing only the cardinality, while providing a specific value for fragment that can be used by schematron or other downstream mechanisms to perform validation tailored to that fragment type.

wendellpiez commented 4 years ago

@brianrufgsa this sounds like an alternative model, subsetting and/or extending (at points) the larger/more general model. As such, perfectly fine and reasonable! Perhaps next step is to sketch out the schema for such a document, or (alternatively) what adjustments to the SSP model would satisfy.

As for the requirement (validating two different kinds of SSP under two different albeit largely overlapping sets of constraints), the way to achieve this is not by trying to support conditionality in the schema content models ("element X is required, except when it's optional"). Much cleaner and easier to build and maintain is a single relaxed base schema (e.g., metadata could be optional in the models where it might be needed under some circumstances) and then relying on process logic (e.g., Schematron) to enforce the constraints that vary across the two (or more) 'SSP subtypes' as necessary. This approach is used in working document processing systems and would support the design you propose.

Until we have a means of implementing Schematron-like tests over in JSON, however (see usnistgov/metaschema#17) this will not be feasible in JSON. While the relaxed metaschema would produce a JSON schema to support either/any subtype of SSP, with no Schematron-equivalent, application logic would be needed to tell and enforce the difference, or to intercept or prevent hybrids/mishmash.

As an exploration / PoC, however, sketching out optional extensions to support this could show a way forward. Where would metadata be needed, besides the root? (Could this be a feature for other scenarios as well?)

As for making things optional that are now required, in order that a fragmentary or partial document could still validate to the schema -- in general I am in favor of that. It sounds like input to Issue #597.

brian-ruf commented 4 years ago

@wendellpiez I don't think it's an alternative model. We want all the syntax to be identical. I like the idea of generating a relaxed schema. Perhaps each model should have a second "fragment" schema generated that enforces all of the same syntax, except for cardinality requirements.

This could be specifically for fragments. I believe it would fully satisfy FedRAMP's needs for file fragments.

Just to pull the thread a little more in terms of your other observations/suggestions: I think metadata will typically (perhaps always) be needed with fragments. There has to be some data about the fragment, so we know what it is, when it was created, why it was extracted, etc. But then, I am suggesting something where the OSCAL root doesn't change.

As in my example above, an inventory fragment would still have <system-security-plan> as the root, with <metadata /> immediately following it. It would also include just the pieces relevant to a system inventory. Those pieces would still need to exist with the same syntax in the same relative positions (same parents, children, and sequence).

Any field/assembly that is present, must still exist in the correct location. The only difference is the author has the ability to omit fields that are normally required (such as <system-characteristics><system-information>), because they are not relevant to system's inventory. These fields would not be required (cardinality of 0 instead of 1) when using the ssp-fragment-schema.

What I don't know is if others have different use cases/needs for fragments where this approach may not work.

NOTE: I still think there needs to be some way to indicate within the syntax that the OSCAL file is a fragment, so processors know to use the fragment schema instead of the full schema. Also to schematron or other business rules can be applied downstream. Worst case, I'd add a FedRAMP extension to the metadata to indicate the file is an inventory fragment. Then tools would know to apply inventory processing to it.

wendellpiez commented 4 years ago

This all makes sense, although I still think it's much easier to produce a single SSP model for all SSPs. (Otherwise an SSP is not an SSP, it's always a "red SSP" or "green SSP".) It can be as relaxed as we need -- then if we wish to distinguish in testing between an SSP that is complete (for purpose X), we can apply the X-complete Schematron, whereas for an SSP that is nominally fragmentary, it can be validated to a 'nominal-fragment' Schematron.

This is easily done by relaxing the metaschema and introducing business rules for the rest, and does not require extending Metaschema semantics or building and testing functionality for every metaschema to produce "relaxed" (everything is optional?) and "strict" versions.

FWIW, one reason the 'conditional schema' is a bear is that it makes validation at parse time impossible, since you don't know until you have parsed the document which schema should be applied. (So a validating parser must be wrapped in another parser before it can validate.)

However, having a fragment flag and then enforcing rules such as "set to x, enforce X rules, set to y, enforce Y rules", is perfectly fine. We could have a relaxed schema and then have the tougher rules enforced using Schematron or logic associated with (and triggered by) this flag.

Even at the catalog layer, there are many validations that we wish to apply in real-world use, that are not provided by the schema. Trying to make the schema do everything is a known pathway to bloat and complexity. Meanwhile the Schematron knife is sharp.

wendellpiez commented 4 years ago

Sprint 26 Update Jan 30 2020

We discussed this in a meeting yesterday. We agreed that some extensions and/or refactoring at the implementation layer will help address the requirements.

We also made Issue usnistgov/metaschema#31, which will enable features we need in the respective schemas (XSD / JSON Schema).

wendellpiez commented 4 years ago

Sprint 27 Update Feb 13 2020

See usnistgov/metaschema#31 and usnistgov/metaschema#39.

In the currently planned design, it will be possible for a metaschema developer to designate any assembly/assemblies as valid root nodes for an instance. This will enable better/cleaner design in the metaschemas at the implementation layer.

Discussion also makes it clear that the design problems are not trivial even given this feature. 🚀

wendellpiez commented 4 years ago

In shaking out and redesigning the Metaschema we are enabling any assembly to be designated as a permissible root, which will help address this requirement. See usnistgov/metaschema#39.

wendellpiez commented 4 years ago

March 12 Update

Current work in Metaschema (usnistgov/metaschema#39) will address this.

brian-ruf commented 4 years ago

I have another use case for this topic. FedRAMP maintains a standard list of citations in an Excel spreadsheet, which applies to all FedRAMP artifacts. This could be expressed in back-matter syntax, but would need to be a stand-alone file, cited by all other FedRAMP OSCAL files.

Right now, OSCAL-based FedRAMP templates have a citation to this file, but its content is not machine-usable. I created an issue in the FedRAMP-automation ( gsa/fedramp-automation#43 ) repo to convert this file to back-matter syntax. It is not valid OSCAL in that form unless we end up implementing fragments, but at least tools could work with it.

wendellpiez commented 4 years ago

When metaschemas are revised to next-gen Metaschema, back-matter can be designated as a document root, to support this.

Under M2, it can already be a root (as can anything else) but we will be tightening that as described above.

However, I am also noticing that the original statement of the requirement on this Issue is more complicated than "we must be able to validate a branch apart from the rest of the tree": it is really about contingent modeling (models that require elements in some scenarios and not in others). That is a broader requirement than simple permitting a particular element to serve as the root of the document.

As I think we discussed, we plan to handle contingent models with constraints that enforce the contingencies (when X is set to Y, there must be a Z, etc.). But this is not a case of that, is it? (I.e., while the back-matter element would be allowed to be at the top, it would look the same as it does elsewhere?)

If it is -- if what you want is not quite standalone back matter, but some other thing (SSP?) with nothing but metadata and back-matter - this is a case where the model could make everything optional (maybe except metadata), while constraints definitions enforce the presence of those things that are required to be present, when they are required to be present, based on an appropriate flag or document property, or a lack thereof.

brian-ruf commented 4 years ago

@david-waltermire-nist @wendellpiez I've managed to find a work-around to my primary use case related to this feature.

I already need a local-definitions assembly in the SAP, SAR, and POA&M for other reasons, which involved having the same component and inventory-item syntax.

For now, the guidance I am providing to FedRAMP CSPs is that when they perform their monthly ConMon delivery (which includes POA&M and system inventory) they have two choices:

  1. Deliver the whole OSCAL-based SSP each month with the OSCAL POA&M
  2. Duplicate all component and inventory-item assemblies from the SSP's system-implementation assembly to the POA&M's local-definition assembly.

The latter choice includes a requirement to also duplicate any role, party, or resource cited from within any duplicated component or inventory-item - similar to how we handle referenced resources for profile resolution.

So instead of extracting the system inventory into its own SSP fragment, the same content becomes embedded in the POA&M instead.

I recognize this may not be 100% consistent with our goals for OSCAL usage, but is a functional work-around until we can fully address the file fragments issue.

sunstonesecure-robert commented 3 years ago

Deliver the whole OSCAL-based SSP each month with the OSCAL POA&M

this is what we are going with for now

ohsh6o commented 3 years ago

@sunstonesecure-robert, is this how you are addressing cncf/sig-security#423?

wendellpiez commented 3 years ago

@david-waltermire-nist perhaps it would be opportune to re-open the question of which elements should be permissible at the root? Currently I believe the XSD and JSON Schema generators behave correctly (keying off define-assembly/root-name in the composed metaschema); however the OSCAL metaschemas do not take much advantage of this feature.

There is a small risk that more extensive use of root-name will expose implementation bugs but maybe that is an argument in favor of doing this.

sunstonesecure-robert commented 3 years ago

@ohsh6o no, sig-security/423 is a different thing - the CNCF Policy WG effort is for producing output from a kubernetes (or other CNCF project) system as oscal-compatible output - likely not natively oscal (certainly not xml) but at least have all the right properties to allow some kubernetes "operator" (code not human) to easily map each field in the PolicyReport output to oscal.

it does however overlap this discussion of where to root things since if said code produces little bits of root-less oscal presumably some code after that will want to insert the oscal chunk into a larger oscal document

my comment above pertains to the process described accurately by @brianrufgsa --- producing the monthly ConMon report including POA&M, inventory, assessments, etc. as oscal. we just dump the whole thing every month. we are pulling in all the parts separately from the various sources, normalizing it via all sorts of janky kludge and then dumping the oscal.

no one currently seems to ACCEPT said oscal yet - the powers that be still want the Excel and Word docs. but that's a different discussion

EDIT: thanks for clarification below! will follow up with folks and confirm!

brian-ruf commented 3 years ago

@sunstonesecure-robert as I understand it FedRAMP will take fully compliant OSCAL with appropriate FedRAMP extensions. We are still ramping up our tool set so we aren't quite pushing for OSCAL yet.

That said, I have always maintained that there needs to be a way to deliver inventory without having to deliver the entire SSP.

I don't really care how that happens as long as it can be a consistent and validated approach.

There is a desire to keep the number of OSCAL models from growing too much, but this is one case where I believe the ideal approach is to have a separate model for components and inventory-items, then allow the SSP model to either include it natively (as it does now), or import the external file (as it does with a profile).

Short of that, I could live with some kind of directive in the SSP syntax to indicate the content is minimalistic as needed to support only components and inventory-items.

Monthly delivery of ConMon content should not require an entire SSP delivery. While both are sensitive, the SSP is more sensitive and shouldn't be delivered more than necessary.

wendellpiez commented 3 years ago

Should we promote this Issue or spin off a new Issue from it? What is the requirement here and has it (not) been addressed via the Metaschema define-assembly/root-name feature? Are there particular elements that are now (not) permitted to be roots, but should (not) be?

The goal of the Issue was to "define a way to validate OSCAL file fragments", while the term 'fragment' was also not defined. However we can indeed nominate any assembly in a model, in its metaschema, to be a valid root of an XML instance. What you would do with such a fragment without metadata, I am not sure (which is why I think this requirement addresses only part of the real problem).

@brianrufgsa feel free to mark this Issue as closable, or clarify if the requirement has not been addressed (by define-assembly/root-name or otherwise). Maybe there needs to be a "deployment model" for an SSP-model variant designed to capture (only) inventory? Does the SSP metaschema need an additional entry point, inventory, with its own metadata and own contents, albeit only part of what a full SSP contains?

ohsh6o commented 3 years ago

@wendellpiez, perhaps we can discuss this briefly in the following week? I want to understand this better as it pertains to GSA/fedramp-automation#35 and update and plan that work accordingly. Thanks!

aj-stein-nist commented 1 year ago

Given the questions around core requirements for this issue and existing comments and labels, I will align the status with "DEFINE Research Needed."

Arminta-Jenkins-NIST commented 1 year ago

At the 11/2 Triage meeting: This isn't something we can currently maintain because it requires substantial changes to the metaschema. Will be closed w/ aged tickets.

wendellpiez commented 1 year ago

Actually this is not true. Metaschema changes would be relatively trivial to enable this.

What would be more work would be the subsequent design and redesign problems, for example publishing the model documentation given more and more valid ways to 'root' your data. How many times do we replicate the documentation for party, for example - or if not replicate it, then break it out and link it.

brian-comply0 commented 1 year ago

Forgetting the general functionality of OSCAL fragments, there is a compelling need to manage inventory content separate from the rest of SSP content. As inventory content and component content goes hand-in-hand, I'd like to strongly suggest that the NIST OSCAL Team either:

For the latter, I would strongly recommend the following for OSCAL 1.x versions:

The aspect of this that most require community conversation is that both SSP/system-characteristics/inventory-item and SSP/control-implementation/implemented-requirement/by-component require component content, which should rightfully be kept up to date in the SSP itself.

Yet similar to the POA&M model, there must be a mechanism for components to be defined in the inventory model itself. in order to handle situations such as:

If there is sufficient community interest and the NIST OSCAL Team agrees, I am happy to create a separate issue for an OSCAL Inventory Model. as well as the metaschema for it.

iMichaela commented 1 year ago

@brian-easyd - Will you be willing to drive the OSCAL-DEFINE spirals researching this issue? Community's input is crucial here.

brian-comply0 commented 1 year ago

@iMichaela while I haven't been able to regularly make the OSCAL-DEFINE meetings recently, if this topic is on the agenda for a given session, I'm happy to lead the discussion, including preparing materials to offer a proposed solution and drive to a decision.

I've had several respected people in the community validate the need to manage inventory content separate from the SSP model itself. (Hopefully some of them will chime in here.) I believe there is a way to meet this need without disrupting the current models for OSCAL 1.x.

When the NIST OSCAL Team is ready to begin planning OSCAL 2.0, we should revisit this topic and possibly take it further.

iMichaela commented 1 year ago

@brian-easyd - thank you. I do not think our team debates the need or importance of this issue. We hope the community will come forward under your leadership to drive the research spirals towards the best proposed solution (prototype) - see the Share Responsibility model prototype or Controls Mapping we posted for review on our reference page. OSCAL-DEFINE repo hosts the research and provides guidance. We are here to support this effort but do not have immediate cycle to address it without community’s support. And your expertise and vision are the best we could dream of.