Accelerated Publishing Initiative

mchelen-gov commented 2 years ago

Background

User Stories

As a content editor, I need a higher limit on content volume capacity, So that I can publish any necessary content on VA.gov.

As a content editor, I need published content to appear on VA.gov as quickly as possible, So that I can get information to users in a timely manner.

As a VFS engineer, I need to easily build frontend applications using CMS content, So that I can display it to VA.gov users.

As a VA.gov user, I need content on VA.gov to load reliably, So that I can depend on VA.gov as a source of information.

Affected users and stakeholders

CMS Content Editors
Developers of VFS teams
VA.gov users

Overall Initiative Goals

Faster time between publishing in CMS and availability on VA.gov
Increased content scaling capacity within the CMS
Content preview within CMS more closely matches production
All use cases of content-build are migrated to other systems and content-build is retired
Reduced difficulty of VFS FE development
Increased reuse of common FE libraries
VA.gov user experience is equivalent or improved

Work board

Accelerated Publishing Github Project for work management.

Project aims and associated work

Next Build and HTML generation

Next Build is the primary code output of the Accelerated Publishing (AP) Initiative. It is a front-end build system built on Next.js that generates HTML for VA.gov. This HTML usually includes content sourced from the VA.gov CMS. In the future, there is opportunity for it to also build the HTML that allows VA.gov's vets-website React applications to be delivered to users.

Next Build is a replacement for Content Build, VA.gov's legacy static HTML generation system. Much of the work of the AP Initiative is to replicate the templates and business logic that Content Build provides in a modern React-based application.

The following epics contain work towards completing Next Build as an HTML generation system for VA.gov:

### HTML generation
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/17701
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/17702
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16308
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14455
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16497
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14712
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14456

Delivery of Next Build content to VA.gov

At initial launch to production, Next Build will act as a static site generator. It will export all CMS HTML for all page types that it is aware of and deploy those to an S3 bucket. This will be done on a cycle continuously during VA business hours (8:00 am - 8:00 pm, M-F), as well as on-demand when certain content is created or updated in the CMS. This is the first phase in a two phase approach.

The end state of Next Build content delivery is to have a persistently running Next.js server that responds to requests for VA.gov content. Rather than building static HTML to deploy to an S3 bucket, Next.js will instead build content on-demand and deliver it to the VA.gov user. This will allow us to only build content that is requested, which will greatly increase the speed at which updated content can be delivered to VA.gov users. This is the key to accelerated publishing of VA.gov content.

### Delivery of Next Build content to VA.gov
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15952
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14469
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14360
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16490
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15926
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/18422

Work in support of VA.gov CMS

One of Next Build's primary use cases is to act as a front-end for the VA.gov CMS. Content that is created, updated, and managed by VA staff is sourced by Next Build and used to generate the CMS 'page' that is delivered to a VA.gov visitor. These systems are separate from each other, but Next Build is how CMS content reaches the public. Some work within the CMS and coordinating the CMS and Next Build is required for these systems to work together.

### Work in support of VA.gov CMS
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/10903
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14468
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14052

Content and application integrity

Work is necessary to ensure that the Next Build application is functioning correctly, and that appropriate people are alerted when it isn't. This can include active monitoring of running application servers as well alerting of build failures. Writing of application tests and the framework for running those tests is gathered here.

Additionally, it is important to test the generated HTML itself to ensure that it is functioning correctly. This work includes checking for accessibility, checking for broken links and other site malfunctions, page weight and usability, and SEO for both search.gov and for private sector search engines.

### Content and application integrity
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14457
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14454

Developer experience and support

One of the prime motivations of the AP Initiative is to improve the front-end development experience at VA.gov, especially for engineers working with CMS content. Just moving to Next.js and React templates accomplishes this on its own. However, it is necessary to make sure that the system is properly documented for engineers, that tools are in place to guide code quality (linting, type checking), and that local or remote development environments are easy to set up and use.

### Developer experience and support
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14467
- [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15488

Historical Sprint documents

Completed tasks by sprint report

See all sprints

```[tasklist] ### Historical Sprint documents - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/17673 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/17672 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/17437 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/17243 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/17245 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16944 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16780 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16510 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16310 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16270 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16027 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15834 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15701 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15486 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15238 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15021 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14903 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14608 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14422 ```

timcosgrove commented 2 years ago

Added 'VA.gov users' as a stakeholder and 'VA.gov user experience is equivalent or improved' as an AC/goal.

cmaeng commented 1 year ago

This effort has been put on pause as of 9/11/22 and will be shifted to the backlog.

TheBoatyMcBoatFace commented 1 year ago

Closing due to AP pause. Will re-open when path forward determined

EWashb commented 1 year ago

This work will resume in July 2023.

EWashb commented 11 months ago

Is the Lagoon Milestone attached to this epic arbitrary?

timcosgrove commented 10 months ago

Is the Lagoon Milestone attached to this epic arbitrary?

It has been removed. We are still refining epics but we've organized our listing of epics to correspond to our current understanding of the milestones that will be delivered within the current PoP.

EWashb commented 8 months ago

Added project to the OCTO DE board. Teams, POs, and VA leadership will look at this initiative-level ticket for updates when needed.

EWashb commented 8 months ago

I tried to link this to the OCTO board, but I've decided to keep this specific initiative at the team level. Excuse the labeling and project noise.

davidconlon commented 8 months ago

@EWashb can we take a look at the structure of these tickets. I think there is a different level of Fidelity that we want to be showing at the OCTO board level

EWashb commented 8 months ago

@davidconlon I spoke with Erica yesterday and backed myself off of the previous attempt to link this. It should be removed now.

timcosgrove commented 3 months ago

About to make significant changes to the structure of the initiative, so archiving the previous structure here.

Initiative state prior to changes April 1, 2024

## Background ### User Story or Problem Statement #### North Stars As a content editor, I need a higher limit on content volume capacity, So that I can publish any necessary content on VA.gov. As a content editor, I need published content to appear on VA.gov as quickly as possible, So that I can get information to users in a timely manner. As a VFS engineer, I need to easily build frontend applications using CMS content, So that I can display it to VA.gov users. As a VA.gov user, I need content on VA.gov to load reliably, So that I can depend on VA.gov as a source of information. ### Affected users and stakeholders * CMS Content Editors * Developers of VFS teams * VA.gov users ```[tasklist] ### Overall Initiative Goals - [ ] Faster publishing time - [ ] Increased content scaling capacity - [ ] Reduced difficulty of VFS FE development - [ ] Increased reuse of common FE libraries - [ ] Fewer applications in the legacy content-build - [ ] Content preview more closely matches production - [ ] VA.gov user experience is equivalent or improved ``` ### Work board Accelerated Publishing is managed via Github Projects. [Our Github Project can be viewed here](https://github.com/orgs/department-of-veterans-affairs/projects/955/views/8). ## MVP Goals & Milestones The current AP contract period of performance (PoP) is July 13, 2023 - April 30, 2024. Within the PoP, our aim is: > **Build static pages for VA.gov with a React-based static generation system and deploy those to VA.gov production incrementally.** * "Incrementally" here specifically means that we only build what is needed, based on what has content has been changed or added. The currently existing system builds and deploys all CMS content. This generates over 20,000 pages each time. This is the primary source of the slowness of the current system. We expect the biggest speed gains to come from only building and deploy content that has been added or updated. * "React-based" as a requirement is there to bring CMS content templating in line with the rest of VA.gov front-end and with modern practices. The MVP for the current contract will be delivered in three milestones. ### Milestone 1: Proof of complete HTML generation by the new system (Completed) **Completed:** Nov 2023 The first milestone focuses on setting up our new build system for generating complete HTML from CMS content with full functional and design parity with the existing system. The first use case is Facility Stories and Story listings, demonstrated by these pages: Story Listing: https://www.va.gov/northern-california-health-care/stories/ Story: https://www.va.gov/northern-california-health-care/stories/congressman-ami-bera-brought-valentine-cards-to-veterans/ During this period, work on Milestone 2 will also be proceeding; see below. ```[tasklist] ### Epics required Milestone 1 completion - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14465 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14739 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/10906 ``` ### Milestone 2: HTML generated by the system is deployed to production, replacing the old system for that content **Estimated delivery:** Jan 2023 The second milestone focuses on delivering the HTML content that is generated by the new system to production, and removing that content from being generated by the old system. This will include: * Setting up the build and deployment pipeline to move content to VA.gov * Ensuring that content produced by the old system and the new system coexist and interoperate without issue It should be noted that at this stage, all content that is managed by the new system **will be built with every build cycle,** similar to how the current system does it. In other words, at this stage we will not just build content that has updated, but all content. During this milestone period, work on Milestone 3 will also be proceeding; see below. ```[tasklist] ### Epics required Milestone 2 completion - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/10903 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14052 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14469 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15952 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14454 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14455 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14457 ``` ### Milestone 3: HTML is built and deployed by the new system 'incrementally'; second product area is built and delivered by the new system **Estimated delivery:** April 2023 The primary focus of the third Milestone is incremental publishing, as defined above. Content will be built in response to updates and additions in the CMS as needed, rather than all content being built every time. This is where we expect to see the most speed gains relative to publishing via the current system. Additionally, we expect to deliver the second product area. Currently we are targeting Benefits Hub content, for example the collection of content re: Disability Compensation Benefits that is reached from this top-level page: https://www.va.gov/disability/ . We are targeting this product area due to its importance to Congressional mandates to provide important Federal content to people with limited English proficiency. It would be inefficient to implement multilingual support in the current system, and so we aim to move Benefits content to the new system and support multilingual publishing there. ```[tasklist] ### Epics for Milestone 3 completion - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14467 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14466 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14360 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15488 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15926 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16307 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16490 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16497 ``` ### Post-MVP work Finally, as we do not expect to migrate the full set of content supported by the CMS within the bounds of the current PoP, we will need to provide backlog for all remaining work and documentation about the system to support whichever team or teams do that migration. Additionally, there are some features of the publishing pipeline which we have identified as out-of-scope for the current PoP, but which would benefit the overall goal of providing content to Veterans as quickly as possible. Those will be backlogged as well. ```[tasklist] ### Post-MVP epics - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14468 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14712 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14456 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16308 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16306 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16515 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16516 ``` ### Historical Sprint documents [Completed tasks by sprint report](https://github.com/orgs/department-of-veterans-affairs/projects/955/views/13) ```[tasklist] ### Historical Sprint documents - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/17673 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/17672 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/17437 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/17243 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/17245 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16944 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16780 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16510 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16310 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16270 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/16027 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15834 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15701 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15486 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15238 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/15021 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14903 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14608 - [ ] https://github.com/department-of-veterans-affairs/va.gov-cms/issues/14422 ```

department-of-veterans-affairs / va.gov-cms