Decouple Newsletter Sign-up Pages From Articles

JamieB-gu commented 10 months ago

@dblatcher has a proposal to stop using Composer to generate articles that act as newsletter sign-up pages, and instead generate these pages dynamically using the newsletters API.

This would require changes on the Content Production side to alter how these pages are created, and changes on the platforms side (WebX, AppsX) to handle these pages in a different way.

dblatcher commented 10 months ago

Current Process

Newsletter designed by editorial using the Newsletters Tool - this results in the newsletter data being added to the Newsletters API (with an initial status of “pending”) and automated emails going to Central Production(CP) asking them to the Tags and Signup Page for the new newsletter.
CP create a “newsletter signup” article in Composer - these articles:
Have the "info/newsletter-sign-up" tag, identifying them as being Signup pages
Have only placeholder body content
The relative path to the “newsletter signup” article (IE the article id with a leading slash) is a added to the newsletter data in using the Newsletters Tool
When the newsletter set up is complete, the status in the API is set to “live”
Web Requests to render these articles have special handling in the code base, due to the "info/newsletter-sign-up" tag:
- The tag informs a service in Frontend to look up the data for the matching newsletter and provide it to the dotcom rendering model
- DCR applies a special article format when rendering the page - the format excludes the article body and instead renders the sign-up form for the newsletter (using the data provided by frontend)

issues with current process

The Newsletter Signup Pages use composer and are treated as “articles” and get stored in CAPI, but aren’t actually Journalism like a ‘real’ article is.
The process for setting up the pages is fairly convoluted and involves some overlapping dependencies - e.g. the form can’t appear on the page until the newsletter is live, the page has to be put live to generate the article ID and so it can be added to the newsletter data - tends to cause confusion
The ‘article style’ URLs for the pages (including the section and publish date) are not memorable
Accounting for a ‘special’ type of article complicates the rendering/platform logic for articles
Missed opportunities for more specific structured data (application/ld+json) - a dedicated page template could have data identifying the newsletter as data object

Proposal

Instead of publishing pseudo articles in composer and, we could have a dedicated feature to generate data-driven sign-up pages based only on data in the newsletter api.

Extend the newsletter data model to include extra fields needed to customize the detail pages - eg the page meta title and keywords, the image ID of the graphic (currently set as the article “main media”) - anything else Editorial could reasonably want to be able to customise.
Add a UI features to the Newsletter Tool allowing users with the right permissions to modify the ‘detail page’ configuration (this could initially be a 'developers only' feature using the existing JSON editor in the tool so the UX doesn't need to be blocking).
For Web:
- Add a controller route to Frontend for rendering ‘newsletter detail’ pages (could use the path already used by the existing ‘all newsletters’ page as the base - EG theguardian.com/email-newsletters/first-edition) - frontend already has the newsletters API data in a cache, so would just need to build a page model to post to DCR
For App
- Add the equivalent feature to request the newsletter data via MAPI, request the HTML from DCAR (unsure of the right terminology here - not familiar with the App). MAPI already has access to the newsletters API for the exisinting sign-up feature.
Add a new method to DCR to consume a data model representing a newsletter details page and return the HTML - most of the components used on the existing article layout for sign-up pages could be repurposed.

When the feature is working:

put in redirects to point the existing article pages to the newsletter detail page (article ids are stored on the newsletter data, so should be easy to arrange
Deprecate the ‘sign-up page path’ property on the newsletters data model, remove from the UI
Remove the frontend code for providing the extra page model data for the sign-up pages
Remove the newsletter sign-up article format from DCR

Side Benefits

It is a general issue that adding/managing pages which are neither Articles or Fronts is difficult and can require obscure workarounds, hard coding and/or custom code in frontend.

This work could provide a test case for generalising page modeling in DCAR platforms and be used a a basis for 'regularising' features like:

football fixtures
about us sections
FAQ pages
election trackers?

dblatcher commented 10 months ago

Possible Dataflow / workflow for data-driven newsletter pages

flowchart TD
Z((Reader))
X((Staff))
B[[Newsletters Tool]]
A[[Newsletters API]]

C[[Frontend]]

D[[DCR]]     
C1[page model]

X <-.edit newsletter data .->B
B .->A
A -.publish\nnewsletters \ndata.->C
C -. cache\nnewsletters.->C

Z --1. request-->C
C --2. handle request-->C1
C1--3. render request--> D
D --4. rendered page-->C
C--5. response---->Z

dblatcher commented 10 months ago

Examples of existing pages https://www.theguardian.com/global/2022/sep/20/sign-up-for-the-first-edition-newsletter-our-free-news-email https://www.theguardian.com/global/2022/sep/20/sign-up-for-the-down-to-earth-newsletter-our-free-environmental-email

Proof of concept branches https://github.com/guardian/dotcom-rendering/tree/dblatcher-newsletter-detail-pages https://github.com/guardian/frontend/tree/dblatcher/newsletter-detail-pages

guardian / dotcom-rendering