Support for OSCAL as configuration input

[drsm79]

Overview

NISTs OSCAL is a public standard used for describing controls, implmentations, systems and their assessment. We have done IBM internal POC work to confirgure Auditree from OSCAL files. This is about how to bring that into the open source project.

Requirements

1. Read standards compliant OSCAL file (model tbc - see below) to configure Auditree.
2. OSCAL input should be stored as evidence for consistency
3. Identifying controls implemented via a fetcher/check should be simple & inkeeping with the spirit of OSCAL
4. OSCAL should not be a requirement for running Auditree, more a nice to have.

You could image a tool that goes the other way (from an Auditree configuration to an OSCAL file). That's interesting, and might help with adoption of OSCAL, but out of scope here.

Approach

We have a few potential approaches, depending on how & where in the life cycle we want to introduce OSCAL. There may be more than one approach used, depending on circumstance. This issue is to discuss & form a plan to go forward.

Component definition

Our current POC uses a component definition to turn on/off fetchers/checks. It does this by scanning the file for control-implementations where type is monitoring, and then scanning the properties of the implementation for a auditree_{fetcher,check}_path. This is then "grafted" onto the configuration repository to produce a full config.

To facilitate this, running auditree becomes a three step process:

1. We retrieve the component defintion via a fetcher, placing it in the locker
2. We run our configuration script to make the appropriate controls.json from the component defintion
3. We run the complete configuration as normal

This has some useful properties:

• it runs as part of normal execution, and is consequently kept current
• combining the configuration in the component defintion with the one from the config repo happens "naturally"
• input OSCAL is stored as evidence
• the fetchers are "constrained" only by their configuration/permissions - e.g. we can "discover" inventory

However, it doesn't help much with configuring the broader Auditree system, for example is says nothing about how often Auditree itself should run (though could if Auditree was defined as a component) nor which Harvest reports are required.

SSP

I think this would look a lot like the component defintion, with the potential further constraint of a specified inventory.

Assessment plan

An assessment plan would allow for "bigger" configuration - for example configuring the frequency at which Auditree is run, or schedule Harvest reports as additional assets/assessment-activities.

I think the workflow would look like:

1. Run an independent tool to configure Auditree from the assessment plan
   • configure tekton/travis/jenkins/circle... pipeline for fetcher/check execution
   • configure tekton/travis/jenkins/circle... pipeline for report execution
2. Retrieve the assessment plan & store as evidence (could use plant or a simple fetcher.
3. Use the assessment plan to configure the fetchers and checks, as in the component definition POC
4. Run the complete fetcher/check config.
5. Run harvest reports on independent schedule, using plant to place results into the locker.

An assessment plan seems more idiomatic, too.

Considerations & open questions

• Auditree should be able to run without OSCAL (as now).
• Should this be baked into the framework (additional CLI option, allowing for multiple), be a separate tool, or perhaps both?
• Where should this code reside, framework + tool?
• Configuration repositories are likely more easily managed than OSCAL, so should at least be used as a baseline.
• What configuration, beyond fetchers/checks to run, should come from OSCAL? Does this require standardisation in fetcher/check development, or could it be embedded JSON blobs?

Security and Privacy

Provide the impact on security and privacy as it relates to the completion of this issue. This level of detail may not be available at the time of issue creation and can be completed at a later time. N/A if not applicable.

Credential management will be a thing, especially in the more complete integration. Obviously creds should not be placed in the OSCAL files, but maybe names/references to them could be (to then be retrieved from Vault or sealed secrets).

Test Plan

Provide the test process that will be followed to adequately verify that the approach above satisfies the requirements provided. This level of detail may not be available at the time of issue creation and can be completed at a later time.

• Define some standard OSCAL files that describe known configurations. Confirm that equivalent configurations are generated by the implementation.

[butler54]

Some thoughts on the comments above:

1) In terms of the steps above under the assessment plan I think step 2.1 would be validate the assessment plan.

Assessment plan validation in this context would be non-trivial - in that you may want to validate the underlying integrity of the associated SSP / component definition** / profile and potentially underlying catalogs.

I think this will be important for a number of reasons.

1. While ideally all OSCAL artifacts will be under version control there is a decent chance some may not be (source dependent). This would give a locker so overall state of a given system could be verified and tracked (think potential changes introduced into the profile and/or a catalog in response to an audit).
2. Even under version control it is unlikely other OSCAL artifacts will have the same degree of integrity checking as what auditree can provide.

If this is the case the validation may make sense to sit in an external tool (perhaps running in conjunction with plant).

If the above statement is a given - I think it would make sense to run the validation as an external tool.

Configuring auditree or not from OSCAL.

• The principle challenge in use of OSCAL is the (despite OSCAL's complexity) relatively simple structure which is supported by the parameters and/or property values: Only string values

• At a first pass this seems extreme, however, OSCAL provides, with each value, 3 scoping / constraining elemenents: ns, class and name

• With these constrains I believe we could configure most current scenarios for auditree. name none of the above elements are required to be unique so lists could occur through group by style operation.

In terms of config vs OSCAL:

• I would adopt an approach where:
• OSCAL is enabled or disabled by a special flag (.oscal file with a property pointing to the assessment plan??)
• If this is the case the configuration will be overwritten by config generated from OSCAL
• This step may or may not be a independent pre-processing tool running in the CICD pipeline.
• Otherwise auditree proceeds as expected.

** in the case where a full SSP exists a component definition is unlikely to be needed to generated / used by Auditree

[drsm79]

re. validation - yeah, I think a check (or set of checks) would make sense. Could do a few different things:

• is the OSCAL valid (passes schema verification)?
• is there at least one fetcher/check in the OSCAL?
• are all the fetchers/checks in the OSCAL available?
• are all the fetchers/checks in the OSCAL running (e.g. configured correctly, not passing)?

I think before we worry about flags etc (I think there are clear ways to add the functionality) the question of whether we should be configuring a run or more of the whole system from OSCAL (e.g. tekton run frequency). That might point to 'built into framework' or 'separate tool' a bit more clearly.

FWIW I think the OSCAL could replace the controls.json, and be provided as an alternate switch to that, possibly allowing multiple (depending on which model is used - e.g. run my SOC2 assessment plan, and my ISO assessment plan, and my PCI assessment plan...). But, that's presupposing the interaction a bit.

[butler54]

I'll add more detail when i'm thinking clearly: The biggest issue I see here is operational / governance segregation. I'm wondering whether OSCAL should configure the objectives of auditree (aka the checks) and the associated fetchers; credentials should be delegated as an implementation detail for auditree.

Appreciate your thoughts - will reply my day time.

[drsm79]

Definitely think creds should be held elsewhere:

Credential management will be a thing, especially in the more complete integration. Obviously creds should not be placed in the OSCAL files, but maybe names/references to them could be (to then be retrieved from Vault or sealed secrets).

Agree there's a scope/ease question around other parameters. Should OSCAL be used just to configure which fetchers/checks are run, with the config repo holding the actual configuration (which is a bit weird) or should it configure the fetchers/checks fully? If the latter, does that make updates awkward?

[butler54]

So the concern I would have is that i'm not sure how often an organisation will want to mess with the upstream OSCAL object.

For example if we assume for a second the profile is storing the configuration of parameters - I would expect that this could become a velocity issues as I would imagine profile changes may need quite rigorous review. (i.e. I think everything in OSCAL could get very awkward).

An in-between might be that from OSCAL you may refer to specific URI's for configuration information for a fetch - which might allow you to keep the content bundled with a profile - or let it default to standard auditree locations. This is getting even messier thought.

[drsm79]

Right, so some kind of reference ("my configuration is in git://foo/bar/config.git") and version?

[butler54]

I think so. To me that makes sense. In the end at some point you move from 'Governance' land to 'ops' land. I think this is that method. It also allows us to hide 'non-compliance relevant' configuration to keep the OSCAL cleaner.