search

publicsuffix / list

The Public Suffix List
https://publicsuffix.org/
Mozilla Public License 2.0
1.96k stars 1.19k forks source link

Update documentation to reflect prioritization and democratic flow of processing requests #982

Closed dnsguru closed 4 years ago

dnsguru commented 4 years ago

Looking for some help in defining an order of processing from @weppos @sleevi to document the factors that impact prioritization and pace of processing requests within our volunteer pool.

The queue and order of PR received for processing is something that I want to try to further define in the documentation.

I'd like to update to the documentation that shows factors impacting processing and potentially identifies the order of processing of updates to the PSL in a manner that should hopefully attract submissions to flow through using the main funnel over other options.

I have reviewed the spectrum of PR and tried to capture the majority of the scenarios that come in to us. The following list is a starting draft to start the discussion of the order of processing (including a category for where it will not be processed)

Glossary:

The following is the list of submission types we receive

Worth noting that these things are unlikely to be addressed

dnsguru commented 4 years ago

Documentation for assistance in roadmap #671

sleevi commented 4 years ago

This is a very complex ontology, and I'm not sure it's correct. For example, I'm unaware of any examples of 3rd party submissions that are DNS validated. Do you have examples?

dnsguru commented 4 years ago

My .fj pr submission was third party with validated DNS

I was thinking maybe if it was a matrix it would be more clear visually, I just dont have even a yellow belt in markdown to make a table for it.

On Tue, Feb 25, 2020, 5:13 PM sleevi notifications@github.com wrote:

This is a very complex ontology, and I'm not sure it's correct. For example, I'm unaware of any examples of 3rd party submissions that are DNS validated. Do you have examples?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/publicsuffix/list/issues/982?email_source=notifications&email_token=AACQTJP3KD3ZDXHQCENF2Y3REW63JA5CNFSM4K3YSM4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEM6L5EY#issuecomment-591183507, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACQTJI6H6VRUFLYO6GBPBLREW63JANCNFSM4K3YSM4A .

sleevi commented 4 years ago

You’re saying someone other than the registry modified the DNS records? 🤨

Thanks, I’ll look more into it

dnsguru commented 4 years ago

You’re saying someone other than the registry modified the DNS records?

No no no no no... I am saying that I, as a third party to the registry, submitted the PR and the DNS was updated accordingly by the registry.

dnsguru commented 4 years ago

Thinking of it as 4 dimensions - First or Third party, Validated DNS with _PSL (or not), Add/Delete/Modify, and what Section (ICANN or PRIVATE)

sleevi commented 4 years ago

No no no no no... I am saying that I, as a third party to the registry, submitted the PR and the DNS was updated accordingly by the registry.

Oh, why would that distinction of who created the PR matter?

All that matters for prioritization is the method of validation, and that should be:

The section shouldn’t make much of a difference, nor who created the PR.

The type definitely matters; additions carry lower risk than removals, and understanding why additions also matters.

dnsguru commented 4 years ago

On Tue, Feb 25, 2020, 7:42 PM sleevi notifications@github.com wrote:

No no no no no... I am saying that I, as a third party to the registry, submitted the PR and the DNS was updated accordingly by the registry.

Oh, why would that distinction of who created the PR matter?

I suggest this matters in the instance of those not validated via DNS

All that matters for prioritization is the method of validation, and that should be:

  • 1P automatic (Including parent domains)
  • 3P by way of publicly documented registry policies
  • 3P by way of registry contacts

I suggest that 1P w/o dns _psl fits in there somewhere, but deprioritized amidst the requests that have been able to validate

The section shouldn’t make much of a difference, nor who created the PR.

I can see that

The type definitely matters; additions carry lower risk than removals, and understanding why additions also matters.

I agree that additions seem less prone to risk than deletions

Modifications can carry high impact as well, but I reckon we could put del/mod in the same bucket

weppos commented 4 years ago

Personally, I apply the following priority:

  1. Requests related to ICANN/IANA suffixes are processed first
  2. Anything in PRIVATE afterwards

Within 2. I tend to give a slightly higher priority to issues that modify or remove existing entries. The reasoning behind that is because those requests are generally easier to handle as the contributor has past experience, hence I can expect a lower LOE.

I'm open to hear alternative approaches, but I'd suggest we don't come with a process that introduces more overhead than the one we try to solve. I like to keep things simple. 😉

dnsguru commented 4 years ago

OK, so I think the answer is that there are some dimensions to how we process these. Tweak or comment:

LoE, impacts the pace of processing, validation or merge of a contribution

This project has a number of contributors, most all of whom are volunteers. To best process things in the most efficient manner, the following are some factors and dimensions that can impact the order and pace of processing by the volunteers.

What type of change is being requested?

Contributor frequency (new vs repeat) submitting party might indicate experience or familiarity that may make the reduce the LoE, or may be posting frequent changes indicating circumstances of higher LoE from closer review.

Validation of submission(s): ALWAYS

This has a range of LoE. These are not mutually exclusive validation techniques, and the bias is the DNS validation as it is difficult or impossible to forge. The other methods create a burden of research for the volunteers and severely impact the pace of processing or prioritization.

Request size and complexity can also impact the LoE and pace of processing:

sleevi commented 4 years ago

Relative to additions, additions are not all equal.

I totally admit, I'm not thrilled with having to ask these questions. They're an unfortunate and necessary evil involved with a static list, which is a result of cookies being scoped to "domains" (boundaries) and not merely hosts.

dnsguru commented 4 years ago

Good material @sleevi

Covers some of what I doc'd and then some. I will integrate it after a night's rest.

On Thu, Feb 27, 2020, 3:57 PM sleevi notifications@github.com wrote:

Relative to additions, additions are not all equal.

  • Does this have the template filled out in a way that answers the common questions
    • Why is it being requested
    • What are example domains (this is particularly important with respect to wildcards and doing the right thing)
  • How many domains are being requested
  • Are the domains being requested for internal purposes (e.g. organization Foo wants their dev/staging servers, which only they use/access) or for general utility
    • Basically, does every user shipping the PSL benefit from the added size/processing?
  • What's the registration period for the domain (i.e. the risk of churning)

I totally admit, I'm not thrilled with having to ask these questions. They're an unfortunate and necessary evil involved with a static list, which is a result of cookies being scoped to "domains" (boundaries) and not merely hosts.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/publicsuffix/list/issues/982?email_source=notifications&email_token=AACQTJPAVC6GYRISOZZJYMDRFBHNZA5CNFSM4K3YSM4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENGNMPI#issuecomment-592238141, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACQTJJEMKE6XPBOJSYWEP3RFBHNZANCNFSM4K3YSM4A .

dnsguru commented 4 years ago

I appreciated the inputs @weppos and @sleevi. I want to keep it simple, but we have to also recognize that many of the people submitting issues or PR are not familiar with some of the factors that can reduce or increase the success and time requirements of them.

Here is the latest, incorporating more inspired from the thread and our empirical experience, in the hopes we're helping users better understand how they can best compose their PR by providing us important information, and have a sense for factors that can impact the time or success.

New Version, Below

LoE, impacts the pace of processing, validation or merge of a contribution

This project has a number of contributors, most all of whom are volunteers. To best process things in the most efficient manner, the following are some factors and dimensions that can impact the order and pace of processing by the volunteers.

What type of change is being requested?

All additions secnarios are not equal, and there are factors that can cause things to take longer or not happen (see "Information furnished in the PR template")

Contributor frequency (new vs repeat) submitting party might indicate experience or familiarity that may make the reduce the LoE, or may be posting frequent changes indicating circumstances of higher LoE from closer review.

Information furnished in the PR template (THOROUGH)

Questions and clarifications, research or review may or may not be needed, based upon the completeness, clarity, and accuracy of the submission.

Validation of submission(s): (ALWAYS)

This has a range of LoE. These are not mutually exclusive validation techniques, and the bias is the DNS validation as it is difficult or impossible to forge. The other methods create a burden of research for the volunteers and severely impact the pace of processing or prioritization.

The first one is the most crucial. The other two are helpful. Having all three makes things go VERY quickly in the ICANN section.

Request size and complexity can also impact the LoE and pace of processing:

dnsguru commented 4 years ago

Updated the Guidelines based upon this issue. @sleevi @weppos take a look please.

weppos commented 4 years ago

Additions typically require lower LoE than Modifications/Deletions Modifications/Deletions are higher LoE

Can you clarify? It's actually exactly the opposite for me. It's faster to deal with a modification than a brand new request. In most cases, same apply to deletion.

dnsguru commented 4 years ago

Could you propose some alternative text?

This was me attempting to reword the following from @sleevi

The type definitely matters; additions carry lower risk than removals, and understanding why additions also matters.

Perhaps I could word my incorporation of that differently, as I had mashed deletes and modifications together, and may also be confusing "risk" with "loe"

I think mods from the same party that had previously submitted PR for same string might carry a simpler validation of the requestor.

So I see your point, and in sleevi's, he wants to ensure that the request itself has been considered.

In the end, I want the documentation to be accurate but don't want to overthink it.

On Mon, Mar 2, 2020, 6:25 AM Simone Carletti notifications@github.com wrote:

Additions typically require lower LoE than Modifications/Deletions Modifications/Deletions are higher LoE

Can you clarify? It's actually exactly the opposite for me. It's faster to deal with a modification than a brand new request. In most cases, same apply to deletion.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/publicsuffix/list/issues/982?email_source=notifications&email_token=AACQTJP4CL4QK3KR23CFMBLRFO6U3A5CNFSM4K3YSM4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENPPXSQ#issuecomment-593427402, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACQTJKQJQDEBMRTNPRXF43RFO6U3ANCNFSM4K3YSM4A .

weppos commented 4 years ago

Perhaps I need more context from @sleevi . From my experience:

  1. An addition is higher LOE because often comes with an education effort. I spend extra time looking into potential side effects the submitter did not consider, sometimes teaching the submitter about incorrect submissions or unexpected behaviors.
  2. Modifications (in terms of editing an existing entry or adding rules to an existing block) have lower LOE to me because I generally can skip the educational aspect as the submitter is in most cases already knowledgeable of the process.

I guess at this point I'm not sure whether stating the LOE in the guidelines has any valid interpretation from the eyes of the submitter. In other words, how is this information relevant?

sleevi commented 4 years ago

On Tue, Mar 3, 2020 at 7:44 AM Simone Carletti notifications@github.com wrote:

Perhaps I need more context from @sleevi https://github.com/sleevi . From my experience:

  1. An addition is higher LOE because often comes with an education effort. I spend extra time looking into potential side effects the submitter did not consider, sometimes teaching the submitter about incorrect submissions or unexpected behaviors.
  2. Modifications (in terms of editing an existing entry or adding rules to an existing block) have lower LOE to me because I generally can skip the educational aspect as the submitter is in most cases already knowledgeable of the process.

I saw those as additions. Modifications were modifications to actual entries - e.g. adding or removing wildcards.

I agree additions carry much less risk (and any risk is, well, tough for the domain holder), while removals and modifications carry more risk to end users, which I very much care about. Loss of privacy/security is, to me, worse than denial of service.

I guess at this point I'm not sure whether stating the LOE in the guidelines has any valid interpretation from the eyes of the submitter. In other words, how is this information relevant?

agreed. I didn’t realize this was being proposed for the Guidelines, vs informal documentation. I’ve been hesitant to say something, as I worry it would seem like I’m adding incessant stop-energy, but I’m not a fan 😅😞

dnsguru commented 4 years ago

To me, I think the goal had been to encourage complete entries where all the steps were performed and have thorough information included with each PR, and to help submitters appreciate factors that might contribute to how fast or slow things happen - in the amount of time it takes to process their requests.

If there is a better way to state something, or there is a disagreement with what was written, I do not have to be the only editor on these :)

On Tue, Mar 3, 2020, 4:59 AM sleevi notifications@github.com wrote:

On Tue, Mar 3, 2020 at 7:44 AM Simone Carletti notifications@github.com wrote:

Perhaps I need more context from @sleevi https://github.com/sleevi . From my experience:

  1. An addition is higher LOE because often comes with an education effort. I spend extra time looking into potential side effects the submitter did not consider, sometimes teaching the submitter about incorrect submissions or unexpected behaviors.
  2. Modifications (in terms of editing an existing entry or adding rules to an existing block) have lower LOE to me because I generally can skip the educational aspect as the submitter is in most cases already knowledgeable of the process.

I saw those as additions. Modifications were modifications to actual entries - e.g. adding or removing wildcards.

I agree additions carry much less risk (and any risk is, well, tough for the domain holder), while removals and modifications carry more risk to end users, which I very much care about. Loss of privacy/security is, to me, worse than denial of service.

I guess at this point I'm not sure whether stating the LOE in the guidelines has any valid interpretation from the eyes of the submitter. In other words, how is this information relevant?

agreed. I didn’t realize this was being proposed for the Guidelines, vs informal documentation. I’ve been hesitant to say something, as I worry it would seem like I’m adding incessant stop-energy, but I’m not a fan 😅😞

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/publicsuffix/list/issues/982?email_source=notifications&email_token=AACQTJKQTTZQ2YQ6YOAFVKDRFT5LRA5CNFSM4K3YSM4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENTMLGI#issuecomment-593937817, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACQTJPZFCLQGR3NXVD67VLRFT5LRANCNFSM4K3YSM4A .

dnsguru commented 4 years ago

Gonna close this one for now. Seems reasonably well addressed in the update I posted.