Write proposal to improve logic for when broken link checker should fail content release

kevwalsh commented 2 years ago

User Story or Problem Statement

Historical

When all content was code, one broken link was enough to fail the build.

Current state:

In Q1 2021, we started failing the build only when there were mroe there are 10 broken links. It doesn't matter how many pages have broken links.

Proposal (rough version)

Now that we have tens of thousands of CMS-supported pages and 600 active CMS editors, we are often coming close to 10 broken links, we believe that it may be time to raise the limit somewhat, but in a way that makes the logic more aware of Veteran impact.

In many cases, one page can be archived or get an updated URL alias, and cause 10 or more broken links. Or, an editor is working on a handful of pages that interrelate, and we get a race condition where they start publishing the interrelated pages during the content release.

We believe that the build-breaking rules for the broken link checker could be tuned to the needs of the much larger enterprise CMS, while maintaining quality of content for Veterans

Possible changes (not mutually exclusive, just a brainstorm)

Count destinations not sources

Currently the system counts the number of sources of broken links. So one unpublished page (one 404) can result in multiple broken links being counted. Flipping the counter to count the number of broken link destinations would make the count more representative of the actual number of 404's a veteran encounters.

Directory based

Allow X number of broken links per top-level directory (eg within /northern-california-health-care/, allow 5 broken links)
Do not allow more than Y broken links in certain directories (eg /health-care or /disability)

Consider number affected pages as well as number of links

Allow X number of broken links if/when they originate from less than Y pages

Others?

Acceptance Criteria

[ ] Write up design doc
[ ] Circulate with affected stakeholders

Implementation steps

## Design principles Veteran-centered - [ ] `Single source of truth`: Increase reliability and consistency of content on VA.gov by providing a single source of truth. - [x] `Accessible, plain language`: Provide guardrails and guidelines to ensure content quality. - [ ] `Purposely structured content`: Ensure Content API can deliver content whose meaning matches its structure. - [ ] `Content lifecycle governance`: Produce tools, processes and policies to maintain content quality throughout its lifecycle. Editor-centered - [ ] `Purpose-driven`: Create an opportunity to involve the editor community in VA’s mission and content strategy goals. - [ ] `Efficient`: Remove distractions and create clear, straightforward paths to get the job done. - [ ] `Approachable`: Offer friendly guidance over authoritative instruction. - [ ] `Consistent`: Reduce user’s mental load by allowing them to fall back on pattern recognition to complete tasks. - [ ] `Empowering`: Provide clear information to help editors make decisions about their work. ### CMS Team Please leave only the team that will do this work selected. If you're not sure, it's fine to leave both selected. - [ ] `Platform CMS Team` - [x] `Sitewide CMS Team`

swirtSJW commented 2 years ago

Currently the array of broken links is keyed by source

broken-links = [
   'source' => [ destinations ]
   'ann-arbor-health-care/health-services' => [
        '<a href="//www.va.gov/ann-arbor-health-care/programs/intimate-partner-violence-assistance-program">Intimate partner violence assistance program</a>',
      '<a href="//www.va.gov/ann-arbor-health-care/programs/intimate-partner-violence-assistance-program">Intimate partner violence assistance program</a>'
   ],
   'ann-arbor-health-care/locations/adrian-va-clinic' =>
       '<a href="//www.va.gov/ann-arbor-health-care/programs/intimate-partner-violence-assistance-program">Intimate partner violence assistance program</a>'
];

counts 3 times

recommending keying it by destination (the bad href of the link)

broken-links = [
   'destination' => [ sources ],
   '//www.va.gov/ann-arbor-health-care/programs/intimate-partner-violence-assistance-program' => [
     'ann-arbor-health-care/health-services' =>  '<a href="//www.va.gov/ann-arbor-health-care/programs/intimate-partner-violence-assistance-program">Intimate partner violence assistance program</a>',
     'ann-arbor-health-care/locations/adrian-va-clinic' =>
       '<a href="//www.va.gov/ann-arbor-health-care/programs/intimate-partner-violence-assistance-program">Intimate partner violence assistance program</a>'

   ]
];

Counts as one 404

ndouglas commented 10 months ago

We're not currently under a substantial amount of pressure because this limit has been raised.

department-of-veterans-affairs / va.gov-cms