FedRAMP external constraints validating by-component at wrong layer

aj-stein-gsa commented 1 week ago

This relates to ...

[X] the FedRAMP OSCAL Validations

What happened?

In the FedRAMP OSCAL Documentation it outlines that by-component elements should be at the statements level (control-implementation>implemented-requirements>statements).

We have our OSCAL formatted as outlined in the documentation, but when validating using the enhanced oscal-cli and the fedramp-external-constraints.xml, it flags this as an incorrect structure. It instead gives the following errors, which suggests that these by-component elements should be at the implemented-requirements level rather than statements.

We were hoping you could help us identify whether this is a bug, or a formatting issue with our OSCAL. Here is a snippet of the OSCAL that is causing these validation errors:

"implemented-requirements":[
  {
    "uuid":"5b55e601-fa5c-58fc-9596-f8e4ee1cccfc",
    "control-id":"ac-3",
    "props":[
        {
            "name":"control-origination",
            "ns":"https://fedramp.gov/ns/oscal",
            "value":"sp-corporate"
        },
        {
            "name":"control-origination",
            "ns":"https://fedramp.gov/ns/oscal",
            "value":"sp-system"
        },
        {
            "name":"control-origination",
            "ns":"https://fedramp.gov/ns/oscal",
            "value":"customer-configured"
        },
        {
            "name":"implementation-status",
            "ns":"https://fedramp.gov/ns/oscal",
            "value":"implemented"
        }
    ],
    "statements":[
        {
            "statement-id":"ac-3_smt",
            "uuid":"554572df-8a52-54da-803c-170d246f6c3b",
            "by-components":[
                {
                    "component-uuid":"c0e9b4ab-7f2e-54da-9cb3-72894240cc3f",
                    "uuid":"f9086f0c-a65d-597c-9c59-88cf02b30c27",
                    "description":"Private Implementation details and description for the following control statement: AC-03",
                    "implementation-status":{
                        "state":"implemented"
                    },
                    "export":{
                        "provided":[
                            {
                                "uuid":"c39e10b2-28ae-586c-9a2a-93c2983d57c7",
                                "description":"<p>This is what is shared with the customer on export, and what the customer configures<br />\n</p>"
                            }
                        ]
                    }
                }
            ]
        }
    ]
  }
]

Relevant log output

How do we replicate this issue?

Rerun the oscal-cli with an example SSP provided by the reporting user out of band (via email)
Locate and confirm if the issue is reproducible or not

Where, exactly?

Validations
- for where exactly a <by-component/> SHOULD or MUST be defined in one or implemented requirements
Documentation on automate.fedramp.gov
Possible alignment with documentation from NIST at pages.nist.gov/OSCAL, if applicable given confirmation and resolution

Other relevant details

Originally posted at https://github.com/metaschema-framework/oscal-cli/issues/55, but GitHub does not permit automatically transferring issues across organizations. I recreated this one manually for @Telos-sa.

aj-stein-gsa commented 1 week ago

Per discussion with @david-waltermire, we need to sync offline on the following:

Is the upstream NIST requirement correct and valid for FedRAMP's use case.
Is the FedRAMP requirement in constraints correct in combination with 1.

We received sample data and more context from the users who reported this in a FedRAMP office hours. More to follow.

Rene2mt commented 5 days ago

I think the FedRAMP constraint "missing-response-components" might need to be updated. The constraint should target the statement assembly (e.g., /system-security-plan/control-implementation/implemented-requirement/statement/by-component). We are expecting control responses to have at least 1 statement (e.g., for each control part a, b, c, etc.), that statement must have at least one by-component (this is summarized here - https://automate.fedramp.gov/documentation/ssp/6-security-controls/#response-overview).

I am reviewing the FedRAMP documentation to see if there are scenarios where we expect by-component to be defined directly in the implemented-requirement assembly and will follow up.

david-waltermire commented 2 days ago

I believe this should be replaced with a constraint that checks that for each response point, there is a statement-level by-component entry. If there are cases where no response point exists, then maybe there is a need for the 1 statement at least method.

aj-stein-gsa commented 19 hours ago

Yesterday a group of us confirmed that the constraint, as implemented, is a bug. This bug is largely on me, I reviewed it and did not understand the requirement. The website document is also unclear, so we will have to fix that as well.

We are going to move forward with a bug fix and documentation update now that this bug is confirmed, thanks for the report @Telos-sa.

/cc @Rene2mt and @brian-ruf for setting the record straight yesterday and explaining the obvious to me (Dave got it already per https://github.com/GSA/fedramp-automation/issues/770#issuecomment-2419456270).

GSA / fedramp-automation