OSCAL toolchain: must be freely available and open source

[rafael5]

User Story:

As an OSCAL user, I must be able to continuously and freely manage and modify all the Controls Artifacts from any/all publishers (NIST, ISO, FedRAMP, and many more) as they change over time.

THEREFORE: all OSCAL tools and toolchains must be freely available, nonproprietary, and open source, with all components of the pipeline published on Github for community input.

Problem Statement

OSCAL depends on a proprietary toolchain to convert source artifacts (NIST, ISO, FedRAMP) to the OSCAL form as a "proof of concept".

https://github.com/usnistgov/OSCAL/tree/master/content/fedramp.gov#extraction--conversion-process

" All control information from NIST SP 800-53 revision 4 and all FedRAMP control baseline details are correlated in an MS Access database, which is part of the MS Office 2016 product suite. The FedRAMP profiles are created with MS Access Visual Basic for Applications (VBA) code, which queries the information and creates OSCAL-compliant XML using MSXML Document Object Model (DOM) Version 6. This tool represents a proof-of-concept. Open-source tools may be developed in the future. "

Goals:

1. All tools of the OSCAL toolchain must be freely available, nonproprietary, and published in machine processable form on Github.

2. The entire OSCAL community must be able to freely modify any and all aspects of the OSCAL toolchain.

Dependencies:

All OSCAL tools depend on only freely available, open source tools, including corporate-sponsored open-source tools (such as MS Visual Code Editor with XML/JSON plugins), open-source python libraries, and freeely available document generation tools (Sphinx, AsciiDoctor), etc.

Acceptance Criteria

1. The complete pipeline from the authoritative source (as published by NIST, ISO, FedRAMP) to the target XML/JSON form must be fully automated using exclusively open-source, freely available tools.

2. All code and libraries of any/all OSCAL dependencies should be published and maintained on a public Github.

[anweiss]

@brianrufgsa to comment on this given the example @rafael5 is specifically calling out is the closed-source manner in which the FedRAMP team is currently executing the conversions for their profiles.

@rafael5 with the exception of @brianrufgsa's current FedRAMP conversion process, all of the other conversions should be in the open. @wendellpiez @iMichaela to comment re. the published tools and processes for converting 800-53 to OSCAL.

[rafael5]

Can you send a link to the transformation pipeline from 800-53 to OSCAL?

[iMichaela]

@rafael5 and @anweiss: NIST will provide and maintain SP 800-53 controls and baselines in OSCAL (XML and JSON). The transformation pipeline from MS Word to OSCAL is not for public consumption since the human-readable SP 800-53 document will not be available for the public in MS Word. SP 800-53 OSCAL Catalog and Baselines will be a publically-available, free, standardized format, maintained by NIST. We will also provide free, human-readable, SP 800-53 controls catalog and baselines in pdf, corresponding to the machine-readable documents in OSCAL.

[wendellpiez]

@anweiss @brianrufgsa @rafael5 @iMichaela My own (personal) feeling: the requirement is not unreasonable except the scope is undefined: what is the OSCAL pipeline? We can't guarantee that everyone's OSCAL processing will be open source any more than W3C can guarantee all HTML processing will be open source. What we can publish and maintain as open source will be the "enabling core"; PoC demonstrations of FedRAMP processing for example -- or anyone else's experimental work -- is not intended as part of it.

[trevor-vaughan]

When I read this, I took it as "There should be an open source toolchain for building OSCAL".

Personally, I worry that this will fall into the SCAP trap where there was this expectation that vendors would create a bunch of wonderful tools and this didn't really materialize.

[wendellpiez]

@trevor-vaughan That is fair, and so also is the observation that there are structural catch-22s that must be undone. To your point specifically, the enabling core needs to be more than just a specification and call to arms; it must actually be enabling.

[rafael5]

@wendellpiez [OSCAL] "needs to be more than just a specification and call to arms; it must actually be enabling"

Precisely. To "be enabling" means there is a fully working reference implementation using a basic set of tools that anyone can freely implement and adapt for their use cases. That is a synonym for open-source.

Currently [OSCAL] "This tool represents a proof-of-concept. Open-source tools may be developed in the future"

The question again: What is the plan for an open-source reference implementation?

[iMichaela]

@rafael5 @wendellpiez @trevor-vaughan @anweiss @brianrufgsa (including @david-waltermire-nist ): The problem statement above: "OSCAL depends on a proprietary toolchain to convert source artifacts (NIST, ISO, FedRAMP) to the OSCAL form as a "proof of concept" is what I addressed earlier. Putting aside its importance, the conversation here derailed from 'give us open tools to convert standards in human-readable format (NIST SP 800-53, ISO, FedRAMP, etc) to OSCAL format. NIST cannot re-publish in OSCAL, standards that are not free in their original format. A tool like the 'pipeline' NIST is using to convert SP 800-53 in OSCAL is very sensitive to any tiny change in the original MS Word format of SP 800-53 which is not publically available anyway.
FedRAMP has its own plan of supporting the effort with tools (for internal and external use) but NIST has no involvement with FedRAMP's plans and deliverables. Particular OSCAL-related proofs of concept 'tools' or 'pipeline' were discussed by our team and will be developed with the community's support (since NIST does not have today sufficient internal resources) and will be made available for free. We will announce our plans when finalized. NIST also encourages vendors to develop OSCAL-enabled tools as they find suitable for their customers. The community's input on what is needed to impact the adoption of OSCAL the most to enable its full automation power is very helpful and greatly appreciated.

[rafael5]

Thank you @iMichaela . The publication of NIST standards in any form of machine-processable form is a dramatic improvement over Word and PDF formats. I appreciate now that translating a Word document to a structured form is by definition a human process.

Do you - or anyone on this thread - know if NIST intends to publish any other standards as OSCAL? Or any other machine-processable form?

Note: This gets to a larger concern about NIST itsself. If this translation of one NIST standard from Word was a manual process, how can we expect NIST to manage, maintain, version control, and publish all its controls standards in machine processable form?

[wendellpiez]

@rafael5 we are trying to agree with you. But we can't open-source others' code (much as we can encourage them to do so), and whatever we might plan, we think it's early to promise a reference implementation for a spec that has barely entered the play testing phase.

It is nonetheless encouraging to see efforts to exploit what we've managed to do so far. Clearly, there is an appetite.

[wendellpiez]

One other thought to add to this. When/while planning for software to share, we would be remiss not to plan for a set of canonical tests for conformance (inputs and outputs exercising any processing specified).

This is in part b/c I think we actually need a diversity of demonstrations for different use cases and applications (whatever the core functionality is), and a good comprehensive set of tests would help support such a diversity.

Thanks again for the comments @rafael5 they are appreciated. cc @howieavp76 re: testing and validations.

[iMichaela]

@rafael5 NIST team is committed to deliver SP 800-53 and the baselines of controls in OSCAL for current and upcoming (rev 5) of the catalog the day the catalog will be published as final. OSCAL, by design, supports versioning - please check the catalog's metadata. Document signing will also be added.

[david-waltermire]

@rafael5 OSCAL is defining a robust set of models for exchanging control (catalog), baseline (profile), system security plan (SSP) (implementation), and assessment information. I believe we can separate the notions of content production from content exchange.

At this time, we are focused on providing formats for content exchange. We expect that there are multiple paths for the content production, some of which will be proprietary and closed, while others will be open source. As @wendellpiez mentioned above, we cannot mandate what all content producers do. Where we can add value is by defining standardized content exchange formats.

FWIW, providing open source tooling that supports content production is a worthwhile effort, but that goes beyond what we are trying to accomplish with OSCAL at this point in time. It would be great if others, like yourself, work on addressing the content production aspect. I see the value in a YAML-based content production approach that is open source. Having at least one is a big step in the right direction.

To this end, it might be better to title this issue "Develop an open source OSCAL production toolchain". I would also suggest recasting the problem statement, goals, and acceptance criteria around the production of such an open source tool chain.

As @iMichaela stated above, we are committed to maintaining the OSCAL formats and to deliver SP 800-53 catalogs and baselines in OSCAL catalog and profile formats respectively. Other catalog producers (e.g., COBIT, ISO/IEC 27001) will need to produce OSCAL content for their control catalogs. Other baseline producers (e.g., FedRAMP) will need to maintain their own baselines. We are currently publishing the FedRAMP profiles due to our close collaboration on the OSCAL efforts, but they may choose to move these to a different site at some point in the future.

We simply do not have the resources to maintain all of these catalogs and profiles for these other authorities. Our current focus is to work on educational material (e.g., documentation, tutorials, etc) to help these authorities understand the OSCAL formats enough that they can maintain their own information. Any work you could do to help ease content production would help in this regard.

[david-waltermire]

This issue is philosophical in nature and does not have any specific development actions. While we agree with the philosophy of developing tooling as open source (which we are working on), I am going to close this issue.