[EPIC] Document the Approach to SAP, SAR, and POA&M Syntax Development

[brian-ruf]

User Story:

As an OSCAL syntax modeler, I need to develop syntax for the system assessment plan (SAP), system assessment report (SAR) and plan of actions and milestones (POA&M) using an approach agreeable to FedRAMP and NIST/OSCAL leadership.

Goals:

Document the fundamental concepts, scope, and planned activities for development of preliminary OSCAL syntax in support of FedRAMP SAP, SAR, and POA&M artifacts.

This work will be conducted over a series of issues:

• [x] #588 - Perform an information element inventory
• [x] #617 SAP, SAR and POA&M Syntax Diagrams and Mock-Up
• [x] #621 SAP, SAR, and POA&M Syntax Modeling in Metaschema
• [x] #639 SAP, SAR, and POA&M Sample Files and Syntax Refinement
• Possible other issues may need to be defined later.

Dependencies:

None.

Acceptance Criteria

• [ ] Scope is clear and acceptable to NIST and FedRAMP
• [ ] Approach is clear and acceptable to NIST and FedRAMP
• [ ] Plan is realistic and achievable within the required timeframes

[brian-ruf]

Scope:

This effort will create preliminary OSCAL syntax to address the FedRAMP SAP, SAR and POA&M:

FedRAMP Security Assessment Plan (SAP), including:

• Initial SAP
• Annual SAP
• NOTE : Blank Test Case Workbook (TCW) is derived from 800-53 catalog content (assessment objectives and actions), and will not have separate syntax here.

FedRAMP Security Assessment Report (SAR), including:

• Initial SAR
• Annual SAR
• Test Case Workbook (3PAO-populated Content only) (High, Moderate, Low)
• Risk Exposure Table

Plan of Action and Milestones (POA&M)

• POA&M Template
• Deviation Requests

---

SYNTAX DEVELOPMENT CONCEPTS

1. Must use OSCAL metadata and back-matter, which includes attachments
2. Must make a reasonable attempt to separate core assessment needs from FedRAMP-unique needs
3. Must use OSCAL extensions (prop, part, annotation) with @ns="" for FedRAMP-unique needs
4. Must follow OSCAL Traceability Intentions:
   • Must point to SAR -> SAP -> SSP -> Profile -> catalog
   • SAP points to SSP and pulls all system-specific information (like system description) from SSP.
   • There may be a few minor exceptions. Example, it may still be prudent to provide the system name and/or ID in every model
   • SAR points to SAP, and follows plan. Documents deviations from plan.
   • Example (Blank Test Case Workbook): SAR -> SSP -> Profile (which controls) -> Catalog (objectives, test, inspect, interview)
   • Example (Results Test Case Workbook): SAP (Results-only) -> SAR -> SSP -> Profile (which controls) -> Catalog (objectives, test, inspect, interview)

---

APPROACH

1. Create complete information inventory
2. Reduce to one "source of truth" (example, the system description will appear only in the SSP and be referenced by the SAP and SAR)
3. Identify appropriate placement of syntax:
   • core OSCAL (named element)
   • core OSCAL (prop/part/annotation)
   • OSCAL extension (FedRAMP-specific need)
4. Create OSCAL Metaschema, including metaschema documentation and examples

NOTE: Creating the FedRAMP-specific implementation guide is outside the scope of this effort.

---

ISSUES, CONCERNS, CHALLENGES

• Will need to rely on NIST OSCAL team to update CI/CD for new syntax
• Will need help (guidance and resources) to create unit tests

[brian-ruf]

FedRAMP currently has separate SAP and SAR templates for the initial assessment vs. the annual assessment. They are similar but not identical, and must be analyzed separately. The initial and annual templates will be collapsed into a single OSCAL model for each document type as follows:

Template	Model
Initial SAP	SAP model
Annual SAP
Initial SAR	SAR model
Annual SAR
POA&M Template	POA&M Model
Deviation Request (DR)

[brian-ruf]

FedRAMP (and possibly other frameworks) requires the assessor to document deviations from the SAP in the SAR.

I am taking the approach that the SAP documents what is intended to happen during an assessment, while the SAR documents what actually happened. As a result, most deviations can be generated by comparing an OSCAL-based SAP to an OSCAL-based SAR.

Example 1, Tools:

• SAP lists intended assessment tools to be used.
• SAR captures actual assessment tools used, and links results to each.
• Deviation of tool usage is generated by comparing the two lists.

Example 2, Scope:

• SAP lists intended controls to assess (annual assessments don't test all controls)
• SAR captures actual controls assessed, and links findings to each control.
• Deviation of control scope is generated by comparing the two lists.

This somewhat departs from the OSCAL concept of avoiding duplication, and instead pointing to upstream data; however, in this instances, I believe it is important the SAR explicitly document exactly what did happen and exactly what was used.

Some data duplication will result. but I would liken it to a double-entry accounting system, where all transactions are entered twice in two different accounts, as a form of checks and balances.

Some OSCAL tools might pre-populate the SAR content from the SAP, such as copying the list of intended assessment tools; however, if an assessment tool from the SAP list is not used, there will be no results linked to it. Likewise, if an assessment tool was used, but not listed in the SAP, no results could be linked to it until it was entered into the SAR list. Similar statements can be made of control scope and results.

In other words, the linking of results/findings, force the SAR lists to be accurate, and allow the SAR to be compared to the SAP for deviation analysis.

[aaronlippold]

An example of this upstream data is the inspec results data we use in the heimdall data format work links inspec test results to controls

On Tue, Jan 21, 2020, 10:58 AM Brian Ruf notifications@github.com wrote:

FedRAMP (and possibly other frameworks) requires the SAR document any deviations from the SAP.

I am taking the approach that the SAP documents what is intended to happen, while the SAR documents what actually happened during the assessment. Most deviations can be generated by comparing the SAP to the SAR.

Example 1, Tools:

• SAP lists intended assessment tools to be used.
• SAR captures actual assessment tools used, and links results to each.
• Deviation of tool usage is generated by comparing the two lists.

Example 2, Scope:

• SAP lists intended controls to assess (annual assessments don't test all controls)
• SAR captures actual controls assessed, and links findings to each control.
• Deviation of control scope is generated by comparing the two lists.

This somewhat departs from the OSCAL concept of avoiding duplication, and instead pointing to upstream data; however, in this instances, I believe it is important the SAR explicitly document exactly what did happen and exactly what was used.

Some data duplication will result. but I would liken it to a double-entry accounting system, where all transactions are entered twice in two different accounts, as a form of checks and balances.

Some OSCAL tools might pre-populate the SAR content from the SAP, such as copying the list of intended assessment tools; however, if an assessment tool from the SAP list is not used, there will be no results linked to it. Likewise, if an assessment tool was used, but not listed in the SAP, no results could be linked to it until it was entered into the SAR list. Similar statements can be made of control scope and results.

In other words, the linking of results/findings, force the SAR lists to be accurate, and allow the SAR to be compared to the SAP for deviation analysis.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/usnistgov/OSCAL/issues/589?email_source=notifications&email_token=AALK42FMGATGBUXEOCK5DG3Q64LR5A5CNFSM4KDJVLR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJQH5LI#issuecomment-576749229, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALK42DYRECSGQHWIDR4LZTQ64LR5ANCNFSM4KDJVLRQ .

[brian-ruf]

23-Jan-2020 Status

Nearly finished information inventory, with some preliminary modeling decisions/concepts sketched out.

[brian-ruf]

30-Jan-2020 Status

Inventory complete. Finalizing data type, cardinality, source, and recommendations for OSCAL core vs. FedRAMP extension. Expect to have that wrapped up by COB today.

Next Steps

• Create diagrams to graphically represent data groupings and relationships (similar to entity relationship diagrams [ERD]).
• Create draft of syntax using MD and YAML

[brian-ruf]

6-Feb-2020 Status

Issue #588 (Perform an information element inventory) complete, pending review. Established and started working Issue #617 (SAP, SAR and POA&M Syntax Diagrams and Mock-Up). Worked on high-level diagrams.

NOTE: Identifying core OSCAL syntax vs FedRAMP extension will need to be revisited as work continues.

Next Steps

Go at least one level deeper (more detailed) on relationship diagrams. More if needed. Begin syntax mock-up.

[brian-ruf]

Historic SAR Findings

The FedRAMP SAR carries a notion of past results, which are typically an annual snapshot. One goal of OSCAL is to better enable continuous monitoring. An approach to historic results is needed that can:

• represent snapshots in time as "assessment cycles" (2018 findings, 2019 findings, etc.)
• represent continuous collection of findings with growing history, but no clear snapshot point (similar to logs)
• resiliency to handle incomplete or missing historic information

The ability to handle incomplete or missing historic information is important for

• new systems (first assessment)
• inconsistent legacy transition of historic findings to OSCAL
• scheduled archive or purge of historic findings
• continuous, but incomplete findings. (perimeter components this week, storage arrays next week or IA family of controls this week, AC family next week.)

Possible Solution:

• All findings should have a collection timestamp at a more detailed level of granularity.
• Under the SAR root, instead of a single `findings assembly, multiple findings assemblies could be allowed. Each represents a grouping of findings.

For the assessment cycle approach, there would be one findings assembly for each assessment cycle.

For the continuous approach, the use of this grouping could be more at the tool creator's discretion. For example, a tool generating SAR data could group findings under the findings assembly based on:

• one findings wrapper per result or per component
• time ranges: each hour's worth of findings are grouped into a single findings assembly
• component ranges: storage array findings in one assembly, database findings in another

[brian-ruf]

20-Feb-2020 Status

Still catching up from extended illness. Made some progress on this. Expect to make up time and complete by end of month as originally scheduled.

[brian-ruf]

Discussion point: While FedRAMP will always require a SAP, is it possible that OSCAL will need to support a SAR without a SAP?

The current plan is for SAR to import SSP content via the SAP (SAR -> SAP -> SSP); however, should we also allow/enable a direct connection between the SAR and the SSP (SAR -> SSP)?

With the current approach to syntax modeling, this would be easy to support since the SAR syntax is intended to mimic and expand on the SAP syntax with the SAP being the "intended" assets and activities, while the SAR is the "actual assets and activities (plus findings). Allowing a direct SAR->SSP connection would only lose the ability to compare in intended assessment plan to the actual execution.

This may be important in a continuous assessment situation.

[brian-ruf]

27-Feb-2020 Status

Still catching up after being sick. Nearly finished with issue #617 (details added to that issue). Expect to be finished with diagrams and syntax mock-up early next week and transition to metaschema development at that time.

[brian-ruf]

5-Mar-2020 Status

Almost caught up on phase II. There are several views and tables in the SAR, which are all inter-related, and relate closely to the POA&M.

The Assessment Results assembly in the SAR model is going to be a normalized single set of information, on which the Test Case Workbook, Risk Exposure Table, and POA&M are based. Many of the other tables in the SAR are going to be "views" of this information.

This is proving to be more complicated than expected - especially ensuring all of the data correlates properly. I have a detailed mapping HERE.

I also have a data call out to several seasoned FedRAMP reviewers to validate and double-check my understanding of how all this information flows from the TCW to the RET to the POA&M (plus related tables). That feedback started to come in during the afternoon of 4-Mar, and results in some tweaking to this information.

I will also do a final round of validation with NIST Friday, and then will transition fully to modeling the syntax in the metaschema.

[brian-ruf]

12-Mar-2020 Status

Wrapped up Phase II and started Phase III, which focuses on creating the actual syntax using the NIST metaschema language. In the early stages of Phase III there have been several adjustments to the model.

The Phase II diagrams and syntax mock-ups in issue #617 were updated to reflect these changes, and a summary of the changes can be found in this week's status entry to issue #621.

Next Steps:

• Finish roughing in all SAP, SAR, and POA&M assemblies in metaschema
• Make a pass to focus on cardinality, formal names, descriptions
• Make a pass to focus on prop and annotation placement to ensure these can be extended in all appropriate places.
• Make a pass to ensure title, description, and remarks fields, as well as @id flags are available in all appropriate places.
• Create example files based on the FedRAMP SAP, SAR and POA&M templates to verify the modeling.

[brian-ruf]

19-Mar-2020 Status

Phase III is nearly complete. A full draft of OSCAL syntax for SAP, SAR, and POA&M is complete, and some progress has been made on verifying cardinality, formal names and descriptions; however, more quality reviews are required to ensure those are accurate. The quality reviews should be complete by COB Friday.

Added issue #639 SAP, SAR, and POA&M Sample Files and Syntax Refinement.

NOTE

I was still making updates to the mock-ups and diagrams in issue #617. In an effort to close that issue, diagram updates are now maintained in issue #621. The syntax is modeled enough that mock-up updates are no longer helpful. Will continue to maintain the three major tabs in the Information Inventory for now.

[brian-ruf]

26-Mary-2020 Status

Phase III is complete and ready for review. See issue #621 for links to the work.

External to this, I am starting to work on the FedRAMP guidebooks for FedRAMP-compliant SAP, SAR, and POA&M implementation in OSCAL. I will be developing the examples and sample files in issue #639 in parallel to the guidebook development.

[brian-ruf]

2-April-2020 Status

Wrapped up the metaschema work in issue #621, and transitioning to focus on issue #639.

All three metaschema models passed all automated checks, and have been published to the repo and the web site.

[david-waltermire]

This work is complete.