VA Formulary Advisor: injected header not appearing in staging

[jilladams]

Description

9767 added domains

On the Formulary site, HTML, JS, JSON are built in Nuxt (framework) on top of Vue, which then generates static pages. They are using TeamSite as a page server, not using the actual TeamSite application. Using it as a place to store servable files. We don't know of other teams using the system this way.

When the JS is loaded in head, that's before the generated file output, and may be getting stomped when the header is trying to inject. Injection is not working in staging. May be an issue with being in Staging, but unclear and not possible currently to test without launching the site to prod.

mitigation strategy

For now, Dave has approved sharing the assets to allow the Formulary team to build a version of the site header themselves. We'll need to research / understand which assets are required / how to share them, outside the context of the main site.

I am sure that CSS inheritance rules will make this tricky, so let's discuss a way to make this possible if you have questions / concerns.

Acceptance Criteria

• [X] Compile assets required to display the global header / footer, including image assets, in a way that can be shared (not sure if email or some other method will be required, depending on file sizes)
• [ ] Pair with Ron Major, Ronald.Major@va.gov, to explore getting proxy-rewrite app to successfully inject the global header in the existing Formulary template
• [ ] If discovery requires investment in proxy-rewrite app, and the effort is bigger than points estimated for this ticket, write up findings as a new ticket that can be injected.
• [ ] If discovery determines we can't get proxy-rewrite to work for them, this ticket should track supporting the scraped template option to get fonts/icons working.

CMS Team

Please check the team(s) that will do this work.

• [ ] Program
• [ ] Platform CMS Team
• [ ] Sitewide Crew
• [ ] ⭐️ Sitewide CMS
• [X] ⭐️ Public Websites
• [ ] ⭐️ Facilities
• [ ] ⭐️ User support

[jilladams]

Per convo with Dave today:

• Dave would like to confirm with Chris Johnson this Thurs re: whether our team can/should support injected header past standard TeamSite implementations.
• Sharing assets: If we aren't able to pursue getting injection working properly, we can provide assets so teams can roll their own version of the header / footer, with caution.
• Maintenance: teams rolling their own would then be responsible for monitoring the va.gov header/footer to stay synced with any changes

This ticket blocked til we hear from Dave re: whether to pursue support for this, or just handoff assets. (FYI @allisonlu )

[jilladams]

Per Dave (slack), let's go ahead and share assets. Ticket is unblocked.

I'm sure we'll run into questions, so holler with those.

[ryguyk]

https://department-of-veterans-affairs.github.io/veteran-facing-services-tools/platform/architecture/teamsite/

(The proxy rewrite works by...)

3. A code snippet included in the Teamsite page downloads the proxy-rewrite app which overwrites the Teamsite header and footer with VA.gov content

I wonder about the feasibility of sharing that code snippet for teams not building on Teamsite to include manually.

[jilladams]

Ryan / Allison meeting re: sharing header assets:

• We don't know how much of the header is CSS vs. JS
• We can't pass the React version of the header to plug into Nuxt / Vue, bc ours is not a web component & the two systems don't play nice
• Formulary team can grab HTML & images from source
  • CSS sharing -- needs its own dig. Not clear how discreet the files are, how much is dynamic and could be provided. Megamenu does have its own stylesheet. Would not be easy to bundle.
• Estimated 5 due to ambiguity / file gnarliness. May be smaller if we only opt to share image files

Could we pass them something that could make the TeamSite injection work? We don't know yet. We would need to research.

Bigger question Jill will take to Dave: should we be strategizing a way to make the modernized header/footer accessible to sites not on Drupal, not on TeamSite templates? (Vs. negative incentive to adopt TeamSite.)

Jill / Allison meeting with Formulary team + Bradley Smelley, Project Manager for TeamSite + Formulary team

• Sandeep Kumar will provide the TeamSite template to Formulary team to see if they can integrate it enough that injection Just Works (TM)
• If not, team doesn't intend to build a full version of the megamenu / footer. Just need to include enough branding to pass muster. (Whose muster? Not clear. Formulary team owns getting reviewed / approvals, so outside of PW purview.)
• We are on hold until we hear back from Formulary whether template path + injection worked or not, so flagging this as blocked again.

[jilladams]

Formulary team has opted to adopt the TeamSite template, to enable injection. When that work is complete they will still potentially need help with testing / troubleshooting the injected header. I'm leaving this ticket open til we get confirmation one way or the other, and ticket will likely span sprint boundary. FYI @davidconlon

[jilladams]

Formulary team revived this request via email thread today. My current thinking, which I'll vet with FE folks tomorrow: Provide the proxy-rewrite app code. In theory this provides all of the scripts, markup, & CSS they need. The main issue is that we don't know how to support it really, so would probably need to make someone available to handhold if they have questions, who would need to ramp up on how it works. The question is whether it will Just Run TM for them.

My other probably very bad idea was to scrape head, Header, Footer markup from live site page source & share that + _icons.scss as starting point. That markup dropped into an HTML file gets desktop experience 90% of the way there. When going this route: Things that work:

• Header menu expand / submenus / open / close
• Sign in modal
• Veterans Crisis Line modal open / close / ctas
• Mobile responsiveness-ish
• Search form submission

Things that don't work:

• Menu fonts (helvetica neue)
• Icons
• Mobile responsivness at 767 if you open the sign in modal and close it again. Stuff gets busted.
• Search typeahead

[jilladams]

Options: in order of preference

1. Debug getting proxy-rewrite working outside of TeamSite templates. L(5-8?)
   • Theoretically proxy-rewrite should work by domain. We'd like to understand why it isn't.
   • Work with the Formulary technical team to see their templates / installation, to help debug.
   • Would need to pair with Formulary team. We have tried separately to get copies of the TeamSite template files. Sandeep Kotian / Brad Smelley provided files in SharePoint (only mutually accessible filesharing option since we don't have FTP and they don't have github hosted files), but Sharepoint flags/deletes the JS files as security risks, and the files shared didn't include any HTML template to see how css/js are called.
   • Risk: may not work for unknown unknown reasons we discover during debug.
2. Deliver a Markup template: rip head, header, footer markup & deliver as an HTML file, along with _icons stylesheet & font pointers - S(3)
   • Need to identify which _icons stylesheet to send (there are multiple)
   • Find where Helvetica Neue is being served & give them pointers re: how to apply / load? (FontAwesome, etc)
3. Deliver the header as a web component - XL(13)
   • Very long term idea, would make header interoperable with other non-React JS apps.
   • Dave has already determined this is not an immediate priority for time. Would need to consider:
   • Mobile header
   • Responsive concerns
   • prod testing
   • adoption by existing injected legacy pages
4. Hand off Proxy-rewrite app files - XS (1) if no implementation support required; L-XL if support required
   • Expectation: Formulary team works out implementation from there.
   • If their team isn't ready to work it out, may require more handholding from PW and no longer be least effort option

We also considered the potential of creating an HTML header only template that could allow the header to inject as an iframe. Ultimately rejected bc we realized clicking any nav items from the iframe header would load new page within the iframe.

[jilladams]

Dave agrees & Formulary is bought in on plan:

1. Troubleshoot TeamSite template / header injection: Our team owns the proxy-rewrite app & we'd ultimately like to understand why it's not working, as that could become extensible work to help support other teams in future who aren't using TeamSite in the standard way. In our next sprint (starts 9/12), we'll assign a front-end engineer to work with Ron to pair / get a code-level demo of the template files in play, and see if we can somehow modify the app or template code to get injection working properly. This may require updates to the app itself, so may take slightly more time. We may also find it's a nonstarter / just won't work.

2. Scraped HTML template: I've staged the files in a dead-end branch of vets-website, to work around filesharing problems with email/SharePoint, etc. & have shared the following info with the Formulary team. This branch SHOULD NEVER MERGE to main: https://github.com/department-of-veterans-affairs/vets-website/tree/VACMS-10029-internet-formulary-header-footer/DELETE_TEMP_formulary_header_files.

   • headfoot.html template includes markup for <head>, <header>, <footer> (scraped directly from va.gov page source via browser Developer tools). needs work:
     • Menu fonts
     • Icons
   • proxy-rewrite files/style-consolidated.scss -- This is not a drop in file. On VA.gov, the menu font "Source Sans Pro" is loaded via node modules, & this file is an example of the defined styles, from line 27. Formulary team may need another mechanism to load that font.
   • font-awesome node files - Also not drop in files. The live site uses a font-awesome free node module for icon definitions & styles. The two files provided here are a reference for icon definitions that include all the missing icons.

Formulary team may pursue the template route to unblock their launch while we wait for next sprint to pursue proxy-rewrite fix.

[jilladams]

Sidenote: when trying to get proxy-rewrite working, Ron noted on a helpdesk ticket (side trail that's now closed):

I was able to inject the application via adding scripts and the entry point DIV element into an asp page template, however the styles are conflicting to the point where this is not going to be usable, which I suspect is due to the platforms using different versions of Bootstrap (3.x vs 4.x), and these cannot coexist on a single page.

The proxy-rewrite app isn't using bootstrap, so this is a thing to look at once we start looking into it with them.

[jilladams]

From James gutierrez:

VA Senior Leadership did not approve the stop gap option for the Formulary team and has asked us to work with you on option 1 for the VA Header/Footer. They are still asking for LOE so if you could provide one for best- and worst-case scenarios that would be appreciated.

[jilladams]

Timeboxed to 1 day for code dive, then check in with findings to sort out what we want to do for Dave's awareness. Goal: confine to 1 sprint.

[ryguyk]

From the walkthrough with Ron

Ron is able to get things mostly working locally. The injected-header code is loaded and executes, but some of the resources are blocked by CORS restrictions.

When he pushes his code to staging, the injected-header code continues to load successfully, but it exits execution because the whitelist configuration for his staging url is incorrect.

We need to update to look like this:

    {
      "hostname": "vaww.internet.staging.va.gov",
      "pathnameBeginning": "/formularyadvisor",
      "cookieOnly": true
    },

This will then allow Ron and his team to verify that things work in staging (provided they can work around CORS issues). As far as production is concerned, it seems that this would just work since the page in question will be at va.gov/formularyadvisor, and that path is already accounted for in the configuration.

It's not immediately clear why this configuration or this configuration were added. It should work without them. @jilladams do you have any insight?

[jilladams]

If we can modify the whitelist to make it work, let's do that.

Re: why www/non-www versions of /formulary were added, that was probably a misunderstanding on mine and Allison's parts re: whether paths needed to be explicitly added like subdomains are. We defined it that way in #9767, and she implemented as asked, so in future we can note: we only need to add subdomains and alternate domains, not paths under va.gov, right? If that's the case, we can remove the unnecessary additions.

[ryguyk]

Some Findings

How does the injected header get injected, anyway?

The code responsible for doing the injection is found in proxy-rewrite-entry.jsx. When that code is run in the browser, it effectively looks for an existing header and footer and then replaces those with the new header and footer. It's a fairly basic concept.

But it's basic because that isn't the real magic here. The real magic is how that code makes its way to the browser to begin with.

It's best to work backwards:

1. That file makes its way to the browser because there's a script tag in the <head> of the HTML document:

   <script type="text/javascript" src="https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/proxy-rewrite.entry.js"></script>

   When that script tag is encountered, the browser fetches and executes the file (note: proxy-rewrite-entry.jsx is packaged up and compiled down to proxy-rewrite-entry.js).

2. But that script tag is not part of the HTML generated by the TeamSite page. So how does it get there? When the request for a page like www.va.gov/health is made, it gets routed through va.gov servers, and the script tag shown above is injected into the HTML. Per the README:

   
   This works by -

3. A request is sent to a VA.gov website

4. The request passes through the www.va.gov servers, where a code snippet is added into the page

5. This code snippet is ultimately a link to the production proxy-rewrite bundle, where the proxy-rewrite-entry begins processing the page.

6. If the webpage is listed in the proxy-rewrite-whitelist.json, the site-wide components are rendered onto the page.

So what did we need to do to make this work for the Formulary Advisor site?

1. First, we needed to add an entry to the whitelist file:

   {
     "hostname": "vaww.internet.staging.va.gov",
     "pathnameBeginning": "/formularyadvisor",
     "cookieOnly": false
   },

   Since they are using a different staging subdomain than other sites already accounted for by simply having staging.va.gov in the whitelist, this required a new entry. When this launches on va.gov, that is already accounted for by having www.va.gov in the whitelist.

2. Next, we needed to get the necessary script tags in their HTML. This is where we changed things up, and this is where there are still some lingering questions about whether things will work. It's not entirely clear what allows step 2 above (i.e. code snippet added into the page) to happen, but it is likely outside of our purview. This code injection was definitely not happening when running their site at vaww.internet.staging.va.gov/formularyadvisor. It seems like the use of a different staging subdomain likely changes things. It seems plausible that once this launches at www.va.gov/formularyadvisor that this code injection might happen. It simply remains a mystery.

So - INSTEAD - we simply copied the script tags (proxy-rewrite-entry along with a few others that are required) directly into their HTML page. This way, the files are fetched and executed in the browser just the same as when they are fetched and executed after the script tags are injected. We just put the script tags there directly.

3. Finally, there's an important point that part of what the code does when it injects the header is it first looks for existing items. Specifically, it looks for "Last Updated..." in an existing footer. So, we copied the footer code from other TeamSite pages and we pasted that into the formulary site's HTML:

   
   <div id="footer-effect" class="brand-consolidation-deprecated">
   
           <div class="container" id="footer">

. . . . . . .

                         <p class="small-print">Last updated October 4, 2022</p>
                </div><!--/sub-footer-->

       </div>

#### NOTE:
If it _is_ the case that this code injection happens once this goes live in production, then this could be a conflict. There'd be multiple script tags requesting the same file. This might not be a problem, but it should likely be avoided. This will likely be the same scenario when running local-proxy-rewrite.

[jilladams]

I'll add some of these notes into the Github docs re: supporting injected header. For now: our work here is done to support displaying the injected header on staging. #9768 will track turning on injection in production when it's time.