mozilla / pkipolicy

Documents for Mozilla's PKI policies - certificate root program, etc.
52 stars 21 forks source link

Change PITRA to Point-in-Time assessment in section 8 #151

Closed wthayer closed 4 years ago

wthayer commented 6 years ago

My understanding of a PITRA is that it is an assessment conducted by an auditor prior to an actual audit that is intended to give a CA a private readout on their readiness for a real audit. A PITRA doesn't result in a public audit statement. I believe what we're looking for in policy section 8 is an audit statement for a Point-in-Time assessment.

sleevi commented 6 years ago

In terms of WebTrust criteria, we're talking about SSAE 18 assessments, CSAE 3000/3001 assessments, and ISAE 3000 assessments. WebTrust presently allows for both direct engagements and attestation engagements, but in both cases, the practitioner expresses an opinion on the matter. In SOC terms, a point-in-time closely aligns with a Type 1 report, and a period-of-time closely aligns with a Type 2 report.

With ETSI, it's messier (it always is), as ETSI audits are certification schemes based on the ISO 17065 framework for conformity assessment bodies. There, the relevant requirements on the period of certification are governed by the applicable criteria - in this case, 319 403. 319 403 sets the maximum period as 2 years, but not does not establish a minimum period within it, or the base documents it derives from (c.f. 7.4.6). As such, there's a spectrum of acceptable ranges for the certification audit, and a spectrum of periods acceptable to consider, based on criteria such as auditor's professional obligations for opining on the matter and any relevant local legislation.

BoryanaUri commented 6 years ago

Dear all,

ETSI EN 319 403 defines the audit period for all trust services in general. A full-assessment is performed at least every 2 years, a surveillance audit is performed at least every other year (chapter 7.9 within ETSI EN 319 403) and every security relevant change of the service needs to be notified to the auditors and is evaluated by the auditors whether it requires an additional audit in between the regular audits even before the change is taken into operation. Additionally for PKI issuing public trusted certificate, there is the ETSI TS 119 403-2 which defines exact requirements for PKI included in the Browser Root Stores: https://www.etsi.org/deliver/etsi_ts/119400_119499/11940302/01.01.01_60/ts_11940302v010101p.pdf Furthermore the requirements for all services are defined within the ETSI EN 319 401, ETSI EN 319 411-1 and ETSI EN 319 411-2. Within these standards the different policies are defined and in case of policies for SSL certificates, there is reference to the Baseline Requirements and/or EV Guidelines which are audited. For all PKI which are included in the Browser Root Stores, the ETSI auditors are auditing additionally to ETSI, Baseline Requirements and EV Guidelines, also to the Mozilla and Microsoft Root Store Policies. This means that full audits are performed annual for all PKI issuing public trusted certificates as this requirement is defined within the Browser Root Store Policies, Baseline Requirements, EV Guidelines and ETSI TS 119 403-2: "PTA-4.2-01: A full-surveillance audit shall be conducted no less frequently than annually." and "PTA-4.2.02: Updated audit information shall be provided no less frequently than annually."

In relation to the “period of time” and “point in time” issues, ETSI clearly defines that every audit shall be performed as “period of time” (Clause 7.9, next-to-last sentence: “In addition, a sample of records relating to the operation of TSP over the historical period since the previous audit shall be examined by the auditor”) As the auditors are following all other requirements mentioned above, a period of time is performed, even if the CA does not issue certificates. We believe that related to this issue the Baseline requirements shall be clearly defining what the terms “point in time” and “period of time” means. Then this problem will not exist anymore. So, we really appreciate Wayne’s proposal to review chapter 8 of the Baseline requirements as obviously, at the moment the definition is misleading and there are different interpretations.

@Wayne: We can propose a corresponding ballot on how to change BRG chapter 8. What do you think?

Above that, we do hope, this clarifies the situation.

Best Regards

Clemens Wanko (ACAB'c) Boryana Uri

sleevi commented 6 years ago

@BoryanaUri Isn't that information what I said, only more verbosely?

I don't see why TS 119 403-2 is referenced, given that the EN is the applicable criteria being maintained. I think we're in agreement that 319 403 merely defines the maximum, not the minimum. As it relates to initial assessments, Clause 7.9 does not set a minimum, nor does it do so for ongoing assessments (e.g. one may pursue another audit within days of completing the previous). As I said earlier, it defines the maximum, not the minimum - and the minimum for that period is not governed by ISO 17065 either, but by individual auditors' own ethics as to what constitutes a sufficient period for examination or per-country NAB rules.

However, I also think the discussion about BRs Section 8 is not relevant to this issue - or at least, that's something for the Forum to discuss (and is, based on the last call). This issue is specific to the Mozilla requirements that specify "Point in Time Readiness Assessment", which is a different report than a "Point in Time assurance engagement", the latter of which produces a limited assurance report for public consumption, the former of which produces a report for management's eyes.

wthayer commented 5 years ago

Discussion on m.d.s.p. supported this change.