commoncriteria / application

Protection Profile for Application Software
The Unlicense
9 stars 3 forks source link

Conformance claim. #102

Closed kgal closed 9 years ago

kgal commented 9 years ago

Replace, "The PP requires strict conformance" with...

  In order to be conformant to this PP, a TOE must demonstrate Exact Compliance.
  Exact Compliance, as subset of Strict Compliance as defined by the CC, is defined as the 
  <abbr linkend="ST"/> containing all of the requirements in section 4 of the NDPP, and 
  potentially requirements from Appendix C of the NDPP. 
  While iteration is allowed, no additional requirements (from the CC parts 2 or 3) are allowed to be included in the <abbr linkend="ST"/>.
  Further, no requirements in section 4 of the NDPP are allowed to be omitted.
kgal commented 9 years ago

And fix references for NDPP.

WeeknightMVP commented 9 years ago

"Strict Compliance as defined by the CC" [citation needed] CC references to "compliance" pertain to implementation standards (think in terms of CMMI). The NDPP seems to refer to CC definitions of "conformance" (see APE_CCL and ASE_CCL in Parts 1 and 3 for definitions and requirements, respectively), calling it "compliance" for reasons I can't divine. This appears to yet another case of NIAP reusing an existing CC label to represent another, unrelated concept.

What about conformance of other PPs (e.g. those incorporating [CC: PP module | NIAP: extended package]s)? Exact [CC: conformance | NIAP: compliance] would by this definition forbid their very purpose. NDPP's definition applies to STs, but if you replace the above text you should retain that we expect future [CC: PP module | NIAP: extended package]s to be defined for various application domains, and that these are permitted (by definition) to clarify and/or add security requirements (as described in the first paragraph of Section 1.3).

Summarily, this says that _ST_s, not _TOE_s, must conform exactly, where we define "exact" in terms of including all components that are required (unconditionally, or conditionally based on selections), any that are optional (or objective, a kind of optional), and not introducing any extended components; whereas _PP_s may introduce extended components (e.g. through PP modules).

kbeiii commented 9 years ago

You're correct; "compliance" should be "conformance" to be consistent with the CC language (every little bit helps). We used "compliance" because we weren't very careful when we wrote those words in the errata.

The notion of a PP module is newer than the words in the errata, and we (the US) haven't figured out yet how we're going to apply that to our existing EPs. Exact conformance is a term for STs relative to a PP (or to a set of EPs and a PP), but you're correct; in the EP (or PP module or whatever) it's expected that any clarifications/modifications that need to be made with respect to the base PP are "permitted" in the EP.

Also, I don't know if it's some "github-fu" that I don't know about, but in the e-mail push I got for the above message, there was an additional paragraph that I'm not seeing now that said:

"Now I must ask: Why? Why would we specify in advance that STs may not impose additional, target-specific constraints on a TOE's behaviour (or that if it does, we don't want those constraints advertised as extended component definitions)? I could imagine an argument to the effect of "That would create more work for evaluators that would have to test those requirements." or "That might confuse acquisition authorities mistaken that their ST-extended components would be vetted with the same scrutiny as our PP-extended ones." But that's not our job as security analysts. Our job is (mostly) to specify security requirements common to applications, and assurance activities to test those requirements. What am I overlooking?"

So in the old way of doing business vendors were allowed to create STs without reference to PPs, or add anything they wanted. In the new way (strictly PP-based), we are putting the requirements engineering effort on the TC, including the associated assurance activities. This leads to enhanced (although not perfect) comparability as well as a shot at trying to get consistent evaluation actions for requirements. If we allowed STs to add their own requirements, then we would have to have some method of vetting the assurance activities, and with our existing resources it's easier to not allow that at this time.

WeeknightMVP commented 9 years ago

@kbeiii: I withdrew the last paragraph because it was more of a philosophical discussion than a documentation issue, but thank you for your clarification.

WeeknightMVP commented 9 years ago

Close. It was referring to section labels from the old document, so I made some changes: