search

usnistgov / OSCAL

Open Security Controls Assessment Language (OSCAL)
https://pages.nist.gov/OSCAL/
Other
666 stars 181 forks source link

Support for "Rules" in OSCAL Models #1058

Open aj-stein-nist opened 2 years ago

aj-stein-nist commented 2 years ago

User Story:

As an OSCAL tool developer, in order to ensure my software can document testing requirements that an information system must implement as one part of cumulative control implementation requirements, I would like enhancement to the OSCAL models to more explicitly define the concept of a rule as a first-class citizen. Modifications and new additions to OSCAL to tool developers to build software for users to give specific criteria to test for a specific kind of implementation implied by control requirements, and have such criteria expressed in OSCAL.

Goals:

Dependencies:

N/A

Acceptance Criteria

aj-stein-nist commented 2 years ago

As I commented during your presentation this looks like a good start; hopefully the resulting OSCAL will enable expression of the simple case (e.g. 1 rule --> 1 test) simply.

I think that is simple.

{
    "uuid": "8f8439e4-14cd-4bd9-a1aa-3a12e6ad3684",
    "metadata": {},
    "components": [
        {
            "uuid": "4a8edd59-57c4-4ed1-bf86-4fcd6cfbb768",
            "rule-implementations": [
                {
                    "uuid": "64a4c42c-2591-4305-a497-62078ca6fc75",
                    "testing-scenario-reference": {
                        "uuid": "52fb5d82-5c70-40d3-b8e1-1637022493af",
                        "test-scenario-uuid": "e7e00644-dd4b-409a-ab0c-a7bc76e56963"
                    }
                }
            ]
        }
    ],
    "capabilities": [],
    "rules": [
        {
            "uuid": "e4712689-e7d6-487b-914f-781a2e68ec17",
            "description": "Ensure application servers implement encryption at rest with approved methods and approved algorithms."
        }
    ],
    "tests": [
        {
            "uuid": "bf9416a5-8bd2-4346-9374-d8810f463c9e",
            "description": "Use OpenSCAP to check 1/N disk encryption settings in RHEL 8.x.",
            "comment": "NOTE: yet to be defined elements to structure test executor, arguments, and executor parameters (I don't mean OSCAL params here. This is not complete and important TBD stuff will be here."
        }
    ],
    "testing-scenarios": [
        {
            "uuid": "e7e00644-dd4b-409a-ab0c-a7bc76e56963",
            "rule-uuid": "e4712689-e7d6-487b-914f-781a2e68ec17",
            "condition": {
                "test-reference": {
                    "uuid": "96e443c1-c611-4cd8-8fe8-407044b3af85",
                    "test-uuid": "bf9416a5-8bd2-4346-9374-d8810f463c9e"
                }
            }
        }
    ]
}

That took me maybe 2-4 minutes after tripping up on brace expansion and using uuidgen in shell to generate legit UUIDs for me. To answer your direct question, the specific part you highlight is only at "rules" and below in the code snippet. Could it be simpler? Of course. But it seems pretty unlikely, beyond very simplistic example cases, there will simply be 1:1 rule to test ratio for a scenario for even the most simple cloud services that are components that comprise a system in the real world. At least, real world systems, "simple" cases go beyond 1:1 mappings (even referencing your examples in https://github.com/usnistgov/OSCAL/issues/1058#issuecomment-1083073163, for the Kubernetes OpenSCAP Kubernetes hardening baseline profile: a single aspect or very closely related group of aspects for a given trait, has at least 3 or 4 different independent tests verify it for a k8s cluster; the same is true for a Linux operating system or a custom k8s flavor like Redhat Openshift Container Platform you had referenced in prior examples).

Is there something about this thus far that is untenable or not simple enough? Please let us know.

pburkholder commented 2 years ago

How does one find out about the meetings on this topic?

aj-stein-nist commented 2 years ago

How does one find out about the meetings on this topic?

More, generally, on our community contribution page.

More specifically, since I presume you mean this topic being "about the OSCAL Rules and #1058 work" and not other model topics, I will discuss with the team if there is a way we can give advance notice when we do have certain topics planned. :-)

iMichaela commented 2 years ago

@aj-stein-nist & @pburkholder -- We might be able to also use one of the OSCAL mini-workshop meetings for this topic. Anca will probably touch on it during her talk on June 15 @ 11:00 AM ET Presentation: IBM’s Trestle - compliance as code orchestrator and automation workflow Presenter: Dr. Anca Sailer Meeting URL: https://bluejeans.com/434440986/9212 For additional information about the OSCAL mini-workshop series, please visit: https://pages.nist.gov/OSCAL/learn/presentations/oscal-mini-workshop-series/

degenaro commented 2 years ago

AJ, I think your example is good. Here's a slightly different view from my mind's eye (ie Anca my disagree or not), in trying to keep with what we've done with the current OSCAL thru re-purposing of properties. rule-test-model-proposal.json.gz The idea is that a component implemented-requirement control optionally comprises one or more rules which optionally comprise one or more tests. I could image an optional "test-procedure" that expresses how to do the tests if there is a step-by-step "complex" procedure necessary. For our automations thus far, the tests are simply run. Not shown, but also possible I could image that the rules (and tests) would be directly embedded in implemented requirements rather than referenced by uuids. So far we have 1 rule comprising 1 test, but multiple rules comprising a control. As always, I reserve the right to be wrong, and I may have gone a bit overboard on the optional items comprising rules and tests.

david-waltermire commented 2 years ago

The idea is that a component implemented-requirement control optionally comprises one or more rules which optionally comprise one or more tests.

@degenaro We want to be able to share the same rule implementation (combination or rule and tests) across multiple controls. This is the reason for the testing-scenario. We haven't implemented a construct yet to associate a testing-scenario with a control-implementation.

I could image an optional "test-procedure" that expresses how to do the tests if there is a step-by-step "complex" procedure necessary.

Yes. We have the activity defined in the assessment plan and results. This might be a good way to define a manual test procedure.

For our automations thus far, the tests are simply run. Not shown, but also possible I could image that the rules (and tests) would be directly embedded in implemented requirements rather than referenced by uuids.

I have some concerns with allowing these constructs to be defined both inline and by reference, as this greatly complicates writing code that parses, persists, and processes content. Allowing the content inline makes it marginally easier for content creators at a cost to the tool developer that has to deal with this added complexity. We need to weigh such a decision carefully to make sure we address the equities in the right way.

degenaro commented 2 years ago

Agreed that rule-implementations in AJ's example (or rules my example) can be shared by multiple controls. The point was that testing-scenario may be unnecessary in many cases (in all of our cases so far). Just looking for a way to not impose this extra layer unless wanted. The solution was to 1) add the optional-test-procedure that is the equivalent of AJ's testing-scenario and 2) have rules list the tests that test them. Much less concerned about the inlining, and suggest this because that is how we are doing it today via re-purposed props.

aj-stein-nist commented 2 years ago

@david-waltermire-nist as discussed during our modeling session, I updated the models in https://github.com/aj-stein-nist/OSCAL/commit/8dc7f354b2496b6143b952a40840202ce277a807 to:

aj-stein-nist commented 2 years ago

@david-waltermire-nist in advance of follow-on work this week on rules, per your requested, I collapsed #1160 and #1168 into the feature-rules branch for ongoing development.

aj-stein-nist commented 2 years ago

Per request from participants in the Model Review meeting on June 6, 2022, I am attaching the slides from today's presentation on the rules updates.

OSCAL_Model_Meeting_202206_Rules.pdf

Following up the last modeling meeting, here is the slides with updates on rules on today's model meeting.

OSCAL_Model_Meeting_Rules_202209.pdf

openprivacy commented 1 year ago

Just for grins (and possible collaborations though the referenced project seems to have slowed down) here's a project for turning legislation into code and running a rules engine on that: https://www.digital.nsw.gov.au/article/rules-code-test-learn-repeat

aj-stein-nist commented 1 year ago

Just for grins (and possible collaborations though the referenced project seems to have slowed down) here's a project for turning legislation into code and running a rules engine on that: https://www.digital.nsw.gov.au/article/rules-code-test-learn-repeat

Thank you very much for thinking of our work and providing this feedback, Fen! I reviewed the site and will determine how much I can incorporate into the work (if we can beyond looking at the site; more involvement seems to require we email this group and USG employees interacting with foreign officials is always a challenge, fun trivia).

Also, outside of work, I have been following the Linux Foundation/OSSF Alpha Omega project on their approach to how they intend to perform similar work for wrapping different kinds of attestations. But I largely skim and observe the high-level approach. No direct inputs or inspirations yet.

https://github.com/ossf/alpha-omega/issues/28

It is nice to see a growing, vibrant space around this kind of workflow and process.

degenaro commented 1 year ago

Currently we re-purpose properties to hold rules and checks (aka tests), employing namespace and class. We envision the OSCAL "rules" enhancement to allow all that we can express with properties along with the ability to group sets of properties into one rule (or check) and to relate rules and checks to each other. With that in mind...

degenaro commented 1 year ago

Metaschema links of interest:

aj-stein-nist commented 1 year ago

This work should go back to user research and discovery, so this will be moved back to DEFINE Research Needed. After that, if it is returned to development as-is, we should consider refinement being needed as this epic, as previously used it, is too large re upcoming #1688 reorganization and needs to be broken down into manageable pieces.

Arminta-Jenkins-NIST commented 11 months ago

Reviewed on 11/2 at triage meeting.