Coordination of Amazon submissions to the PSL

[aph3rson]

Based on comments in #1460, #1590, and #1600, I wanted to open this issue to centralize procedural discussions away from individual PRs.

We (AWS) are trying to make improvements in our stance with the PSL, with regards to both our current presence on the list and procedures for adding new suffixes. As a result, we've implemented the following changes to support problems called out by the PSL maintainers.

Quality control

• Suffixes are reviewed internally by both developers and security personnel before they make their way into a PR (draft or otherwise)
• The internal runbook for this process includes provisions to reduce the number of suffixes to the minimal amount required for the service

Aggregation

• Service teams do not submit individual PRs for their service/feature
• Suffixes are batched together on a cadenced schedule, to reduce the number of PRs to review/load on the public PSL maintainers

Attribution

• PRs will come from a repository owned within the @aws GitHub organization (@aws/public-suffix-list)
  • Permission to modify that repository should be understood as approval from AWS to contribute suffixes back to your team
• Users authoring PRs in question will be public members of the @aws organization
• PRs will contain signed commits only, using a GPG key which GitHub has verified for that user
• PRs not submitted through this process can be closed, including currently in-flight ones:

  • 1403 can be closed as superseded by #1590.

  • 1404 can be closed, and will be included in an upcoming on-cadence PR.

  • 1460 can be closed, as we'll handle this separately (see the below section on reorganization)

---

That being said, we still have some outstanding items to agree upon:

Cadence / PR size

• N.B: These are bundled together - longer cadence ➡ larger PR size
• We initially proposed a 2/month PR cadence (see @plygrnd's comment: https://github.com/publicsuffix/list/pull/1460#issuecomment-1018853222)
  • We hadn't received any pushback, and figured it was acceptable
• Longer cadences (e.g. 1/qtr, 1/y) have been proposed, as the above cadence might be too much
  • AWS releases new features+regions often, multiple of each per year - yearly would be too-long a cadence
  • We can agree on quarterly, but that means larger PRs
• We currently have a limited number of services we're trying to onboard - when we increase our internal pressure for all services, suffixes per PR will increase
• We're aware of the propagation delay for the PSL for some consumers (e.g. browsers)
  • We're also aware of runtime PSL consumers
  • Our primary focus is keeping the PSL up-to-date at an agreeable pace

Suffixes per service

• Suffixes per service mentioned is a result of #145
• Only allowing a wildcard in the left-most label means that many AWS services cannot use wildcards
  • DNS structure at AWS aligns uses the increasing-specificity model and shows how domain names/services are owned/operated internally
  • The region is generally at the 3LD in the domain - e.g. resource-abc.service.region.amazonaws.com, meaning a different suffix per region
  • This DNS structure cannot be changed
  • A limited number of services have their own 2LD (e.g. resource-abc.region.supercoolawsservice.com) - these services will use wildcards
• Per notes above, we limit PSL suffixes to only those absolutely required
• AWS operates many regions globally, with a handful of new regions every year
  • We'd then need additional suffixes with every region launch
• We appreciate aligning the spec with major implementations, but this is a side-effect of that change

Reorganization of suffixes

• 1460 initially grouped all of our domains into a single section, without any subdivision.

• The so-called "CentralNic model" (mentioned here: https://github.com/publicsuffix/list/pull/1460#issuecomment-1017955653) is one we find more apropos given the scale of Amazon and AWS as a whole.
  • The only minor difference we might ask for is adding an opaque reference to the comment block - we did this in #1590. This lets us refer back to where these suffixes were requested internally.
• We can tackle this reorg in an on-cadence PR, or an off-cadence one - we leave this decision up to the PSL maintainers.
  • An on-cadence PR will mean less PRs to approve, but the diff of the PR will likely be difficult to parse/rebase if needed.
  • An off-cadence PR will be easier to parse/won't obscure net-new suffixes, but will mean an additional PR to review.

[dnsguru]

s/Nebraska/Seattle/

You might be familiar with this image. We're authentic with ourselves about volunteer cycles involved in maintaining this catalog of etld and would appreciate if the public are also.

[dnsguru]

Reminder: This is open source and volunteer-cycle fueled, and not very well resourced. Persistence / responses take up volunteer time just like review and processing PRs.

Are there abundant cycles within your team at AWS to help field and review PRs to help out with resourcing gaps, such that we might be able to quid-pro-quo and improve our turnarounds?

[vixie]

abundant-- no. available-- can be. i'll look at approvals inside amazon. what's involved on the publicsuffix side in terms of training and process?

[dnsguru]

A lot of the heavy lifting, surprisingly, comes from back and forth dialog with submitters to obtain complete, correctly formed, well validated entries, pushing back to ensure file size modesty, DNS resolver verification, registration term and other review.

We also want to have submitter org != reviewer org (so, Amazon could not really review their own, but could review others).

Help in reviewing PR would unburden cycles elsewhere, which might allow for keeping up better.

We additionally want/need to add lifecycle reviews, removals, and better go automation to catch IANA ccTLDs and IDN ccTLD as they launch into the root.

On Sat, Sep 10, 2022, 3:59 AM paul vixie @.***> wrote:

abundant-- no. available-- can be. i'll look at approvals inside amazon. what's involved on the publicsuffix side in terms of training and process?

— Reply to this email directly, view it on GitHub https://github.com/publicsuffix/list/issues/1605#issuecomment-1242704343, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACQTJLXTIJB6V6P26CUEZTV5RSYBANCNFSM6AAAAAAQB3ARMM . You are receiving this because you commented.Message ID: @.***>

[dnsguru]

@vixie @aph3rson

Another time-consuming aspect of reviewer cycles is when requestors fail to put in place the required TXT records.

Having the preset required TXT records in publicly validated DNS to align with the respective PR will make these requests less back/forth and more efficient.

SPECIFICALLY, I went to verify the in #1590 and there were missing TXT records with the PR in them. I got NXDOMAIN responses on these. see here

[vixie]

is all of this work done by hand? that is, do you have automation that sweeps both existing and requested PSL entries for these TXT RRs. or is it just DiG on the CLI?

[dnsguru]

it depends on the tools at hand and access, but these are done by hand most of the time.

Automation tools would be helpful for validating, although it is important to rotate the public resolver checked as an additional means of entropy in the validation process

On Fri, Sep 23, 2022 at 11:36 PM paul vixie @.***> wrote:

is all of this work done by hand? that is, do you have automation that sweeps both existing and requested PSL entries for these TXT RRs. or is it just DiG on the CLI?

— Reply to this email directly, view it on GitHub https://github.com/publicsuffix/list/issues/1605#issuecomment-1256368653, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACQTJNYDWYNQCBPL3LCMVLV7XE6FANCNFSM6AAAAAAQB3ARMM . You are receiving this because you commented.Message ID: @.***>

[dnsguru]

Just some housekeeping, I believe that we have addressed the primary objectives of this issue.

A loud gratitude to the participants at Amazon who coordinated with and within the AWS to migrate and consolidate Amazon's various listing for putting in the effort internally to validate and organize to company's PSL entries.

This was a great reduction on the tech debt and delay in processing Amazon's requests as they stacked up. We will watch for future Pull Requests and appreciate the collaborative effort in aiding to move the pull requests to a closed state and organizing the entries.