Analytics Report for COVID VA Health Response

[joanneesteban]

Metrics Reporting Request

Description

Who is this request for?

COVID Response for VA.gov:

• Chante L-S
• Clarice C
• Leah B

Primary Stakeholder: Lauren A. + VHA + VA Health Stakeholders

Primary Goal

Build a quick domo prototype dashboard that can incorporate CSV updates weekly while we wait for BigQuery acquisition.

User Story

As a healthcare decision maker, in order to prevent VA system overload and covid spread at medical centers, I need to:

• direct patients to self-service information and chat tools
• divert in-person appointments to telehealth
• quickly scale telehealth appointments
• triage new appointment requests through the above

Additional Product Goals: VHA decisions makers can monitor longitudinal problems, use available VA.gov user metrics to get ahead of triaging and scheduling issues, and monitor usage fairly regularly.

Timeframe for Analytics Requested

Daily analytics beginning February or March 2020.

Analytics to Track Down

Begin with Google Analytics (MVP)

1. Basic google analytics for covid and health content (weekly update w/ daily traffic over last month and entry/exit page info)
   • https://www.va.gov/coronavirus-veteran-frequently-asked-questions
   • https://www.publichealth.va.gov/n-coronavirus (do yall have access to this? I can help with that)
   • Clicks from va.gov covid banner
   • Other relevant health page analytics (would like advice from analytics team on this)

Once we have a framework with GA metrics, begin to dive into Contact Center data:

2. Contact Center Data

• Top reasons for calls to call centers
• How overwhelmed call centers are
• Anything else we should include?

Finally, using VEText data (from Clarice) as a mechanism for delivering the previous data sets:

3. VEText Metrics

• Total # Sent (Attempted): 8,858,481
• Total # Sent (Success): 8,639,450 (97.5%)
• Total # Bounced: 219,031 (2.5%)
• 180,731 messages to numbers “not currently reachable via SMS” (most likely carrier/phone not able to receive messages from short code)
• 7,603 messages to numbers not associated with a mobile phone
• 14,423 messages sent to invalid phone numbers
• Total Opted-Out: 97,273 (1.12%)
• Click through engagement: 2.8 million = 32% engagement (this is extremely high by industry standards)

Additional Comments

"I need indications of when each data point was last updated."

Please leave the following blank

Acceptance Criteria

• [ ] The data has been identified and documented in this ticket - items that we don't have access to are noted with cannot be accessed
• [ ] The metrics are shared back with users.

Definition of Done

• [ ] All appropriate issue tagging is completed
• [ ] All AC completed

[joanneesteban]

First Tasks

Drafts:

1. Grab GA URLs for:
   • FAQ
   • Banner
2. Do we have tracking for this URL? If not, let @clantosswett or @joanneesteban know to triage this question.
3. Brainstorm health analytics that we're currently tracking that might be helpful with the user story.

^ @jonwehausen @amycesal

@clantosswett - any other context to add?

@Kimberley2019 - We won't be looking at Contact Center data until we've gone through an MVP with Google Analytics data. So we'll pull you into this when we're ready, but just to be aware and if you have time, begin brainstorming different data we can pull in.

[joanneesteban]

Creating a conversation in Slack to communicate needs.

[clantosswett]

VCL contact center numbers attached here

VCL Metrics 3-24.xlsx

Also including the topics/themes the VCL is anecdotally tracking as trending:

• How to get proper health care while appointments about being cancelled.
• Fear about not having access to medications.
• Support and therapy groups being cancelled
• If I suspect I am Covid+ what are my next steps?
• If I have been turned away from my doctor and told to self-quarantine, what do I do?
• Confusion from responders about whether to advise connection to telehealth, call VA ER, or get EMS, etc.

[jonwehausen]

Relevant GA Reports for COVID

• COVID FAQ Page Navigation Clicks
• Homepage banner clicks
• FAQ Page Expander Text
• Login Events

[joanneesteban]

Other Updates

VCL data was added to dashboard on 3/25/20. Will need reconfiguration as the table has been added, but is not lined up with the other metrics.

[amycesal]

Hey @jonwehausen Can you get the call Veterans Crisis Line spreadsheet to display as a bar graph like this:

The data needs to be manipulated to calculate the call total that's covid related.

[joanneesteban]

Follow Up

Data we currently have connected to Domo:

• FAQ Navigation Events
• VCL March 1-23
• Covid Homepage Banner Clicks

Data we need to connect:

• publichealth.va.gov (GA Implementation)(Previous/Next Page Path)

Baselines vs. Now:

• General VA.gov Traffic
• Health Services Traffic

How to Visualize & Filter:

• Covid Homepage Banner Clicks
• FAQ Navigation Events (Maybe top 10% of events to filter down & visualize high traffic?)
• publichealth.va.gov

Add annotations: (Do we have access to a VA source for this, @clantosswett ?)

• 1st case in USA
• 100th case in USA
• Federal emergency declaration
• WHO pandemic announcement
• 1st state lockdown

@amycesal @jonwehausen

[amycesal]

Key dates for annotations:

JAN. 21 - First US confirmed cases MARCH 13 - US declares a national emergency

Sources: https://www.nytimes.com/article/coronavirus-timeline.html

[joanneesteban]

Implementation To Do

@jonwehausen

• Add key date annotations above (if it makes sense)
• Adjust VCL chart to Amy's sketch
• Filter FAQ top 10 clicks & add to dashboard
• Add publichealth.va.gov metrics
• Add general VA.gov traffic (then vs. now)

@joanneesteban

• Add VCL annotations/definitions
• Add WBC vs All VA.gov definitions

@nedierecel

• Emails from Gov Delivery?

@amycesal

• Visualize publichealth.va.gov metrics
• Visualize comparative traffic

Questions for Lauren...

• What are the specific calls to actions?

Next

Before & After Traffic for

• Facility Locator
• Secure Messaging (MyHealtheVet)
• Prescription Refills
• Telehealth (telehealth.va.gov) - who owns this property?
• Connected Care (connectedcare.va.gov) - who owns this property?
• (Proactive) Cancelled Appointments

VAOS Weekly Report 19MAR2020.pptx

Later

• Subdomains for health services

• Potentially (if traffic is high) - where are users going?

• What impacts are school closures having on EDU benefits?

[jonwehausen]

Updates EOD 3/26/2020

• Constructed charts of high-level overall va.gov metrics
• Adjusted call center metrics to closest format could get to @amycesal's sketch
• Added annotations to a few
• Want to discuss a different approach to the FAQ page metrics
• Hitting a snag with filtering data down on publichealth.va.gov (seeking assistance)

[joanneesteban]

Touchups to Current MVP

Thank you for adding all that data, @jonwehausen !! 🙌🏼

@amycesal

A few follow ups

• Adding in some of these charts for All VA.gov data is to show the baselines vs. now, or is the coronavirus changing our numbers? If so, where? First thought would be the best way to visualize is the same way the contact center numbers should be visualized. Thoughts?
• Edited some of the titles - Jon, if you can make sure they're still accurate
• Can you also add the source links to the GA charts?
• Each of the charts has a different time range - is there a way to align these better?
• Is it possible to get the healthcare metrics up?
  • What are the blockers on the publichealth.va.gov metrics?
• Homepage Banner Clicks: Where did the legend go? I see that it's enabled in the analyzer, but it's not coming up on the dashboard.
• VCL (Contact Center) chart: I actually think the Covid-related calls as the bottom steps are easier to read. The blue "Handled Contacts" in the legend is confusing, though, because you don't see the entire blue column until you hover over it.
• Avg. Session Duration vs. Avg. Time on Page: It's interesting to look at, but I'm not sure this metric is useful. I'm not sure what I'd do with this info. Also, changed the summary number to AVG instead of Sum.
• Total Users vs. New Users: Changed the summary number to max instead of sum. Not sure if it's just my page, but I can't see the annotations on this chart when it's open on the dashboard.
• Sitewide metrics: Changed the summary number to max new users instead of sum users. Amy, can you help with what this dataset can be useful for?

[KevinMHoffmanUSDS]

Thank for that Domo demo, @joanneesteban and team! Kudos on how quickly you've pulled all this together, and to the whole team for their work. Here's some feedback in a DO, TRY, CONSIDER format.

DO In order to get this to a sharable state, we'll need to add

• FAQ page GA metrics we discussed
• The topics listing that @clantosswett is gathering
• For now, the VetText information incorporated as page text (we can revisit later)
• Anything else we mentioned on the call that I may have left off

TRY I would use the original list that @leahbannon identified, that you put above in this ticket, as a guide for what we're hoping can get in there. If we can't include the public health site GA metrics at this time, that's okay, but we'll need to incorporate eventually. If Big Query would help unblock that, let me know.

CONSIDER I would suggest that you add health care benefit hub GA metrics. (/health, top sub-pages at /health and possibly applications for health care conversions, just as an example of how an application could be integrated.)

As soon as these tasks are complete, we'll share this with Lauren and other stakeholders for feedback and start to invite the right people to the dashboard to begin using it, and further iterating.

[joanneesteban]

@amycesal @jonwehausen Please see Kevin's comment above!

Follow Up

• Jon - can you add the FAQ navigation card to the dashboard? Also, would the publichealth.va.gov and the health care benefit hubs data be more easily retrievable or unblocked by BigQuery?

• [x] Added the VEText data as static text to the dashboard

[jonwehausen]

@joanneesteban Top 10 FAQ Page Navigations have been added

To your question, with BigQuery -- we could be connecting DOMO to BigQuery -- the snag we're currently hitting is with the GA DOMO connector, assuming we don't hit any connector-related block, we would be able to get that data quickly in the dashboard.

[joanneesteban]

Message from @clantosswett :

1. Are the metrics that VCL is reporting a good template for what we want from other call centers? What other metrics would we want delivered?

   • Resolution of the call (transfer, resolved, any additional actions taken)

2. The themes section is the anecdotal information on disposition of the calls. It is heartbreaking and I think this type of information is important to share out. I’m going to look into the VA311 IVR for quantitative metrics on what users seem to be after in their questions. We could also ask call centers to run reports on most referenced Covid-19 articles in a day as a way to work backwards and determine what veterans are calling in about.

3. If we were to generate a daily contact center report that compiled similar metrics across many contact centers, what should that look like to make it easy to quickly update the dashboard?

---

@clantosswett how would you want the themes and anectodal information shown?

Excel is probably the best way to deliver the qualitative information. @nedierecel what was the best way to edit the ForeSee data? It looks like we can do an Excel plug-in, but it can only be run on a Window's machine. Potentially can be owned by Chante (so you can just add the data to your Excel doc daily and then refresh. We can do this using the Google Sheets plug-in as well.

[joanneesteban]

@jonwehausen

The updated VCL sheet. I didn't want to mess with yours. VCL.Metrics.3-27 with Total Covid Calls.xlsx

[joanneesteban]

@jonwehausen - can we define New Users & Total Users on the chart?

[joanneesteban]

Update

Dashboard link.

Now Tasks

Done

• Static VEText and Contact Center info on the dashboard
• FAQ Exits on the dashboard

Doing

• Adding FAQ entries as screenshot with link
• QA-ing design
• QA-ing data definitions
• Chante & Jon to coordinate how to upload daily Contact Center data

Blockers

• publichealth.va.gov and FAQ page entries are not linking on the dashboard. This is a Domo issue and we are trying to escalate the fix with the Domo team.

Next

• Kimberley is working on helpful contact center data to add
• Brian & Jon to coordinate health hub benefits additions

  (/health, top sub-pages at /health and possibly applications for health care conversions, just as an example of how an application could be integrated.)

[jonwehausen]

Current GA DOMO Connector Issue

• We have a support case opened with DOMO on the following issue: We are unable to pull in direct publichealth.va.gov/n-coronavirus data with our traditional filtering parameters. When attempting to filter to just that specific page of data, we are receiving an invalid formatting error that we have not received in the past under similar conditions.
• In the event we were to leverage the DOMO BigQuery connector, there's a good chance we wouldn't receive this issue since we would utilize a different connector and we would also have a higher quota limit

[clantosswett]

Attached are IVR metrics from the first three days it went live. This is the IVR offering on My VA311 and it went live Friday morning (3/27).

Covid-19 IVR Metrics 3-30.xlsx

[joanneesteban]

FYI @jonwehausen

Added the IVR dataset. Can you connect with @clantosswett on how to update the contact center metrics?

@amycesal the new data is a table right now, but can you visualize how it should look like for @jonwehausen ?

[jonwehausen]

@clantosswett and I are set to meet at 4:45 EST today to discuss.

[joanneesteban]

Thanks, @jonwehausen for updating the FAQ entrances & publichealth.va.gov charts!

Can you elaborate on here why we are able to get the data but still have an issue with the connection?

Other question: what is the difference between entrances and (users + new users)?

[jonwehausen]

@joanneesteban

We were able to get publichealth data to come back on a large dataset ingestion of the All VA.gov coupled with an ETL filtering for publichealth.va.gov. We will need to attempt to do the same thing with Modernized content. That said, once we get the GA Filtering issue resolved, we can get to the data we need more quickly and don't need to do the manual ETL.

Great question!

• Entrances increments when the user has began their session on the page provided (in this case publichealth.va.gov/n-coronavirus).
• New users are users who have never been to va.gov prior according to Google.
• Users are ALL users to VA.gov - new or returning

[amycesal]

I did a bunch of work on this today. I moved all the charts, so there's groupings of

• call center
• va.gov
• publichealth.va.gov
• VEText data

I added a notes section for definitions of things to be next to the charts for easy interpretation. I re-labeled a bunch of charts to hopefully make them more understandable.

There's a couple of tables still in there. One is just more metrics, the chart above it shows some of them. The other table has 3 days of data and multiple points? I think we need to see how often we get this contact center data before showing it in a chart. I threw a heatmap over it for now.

[jonwehausen]

Dashboard updates have been:

• Re-ran dataset to have homepage banner up-to-date as of yesterday
• Update formatting per Amy's requests (some simple as a label update, others required more advanced beastmode to format)

Current block:

• Retrieving entry data into FAQ page -- most recently hit quota limit

[joanneesteban]

Thanks, @jonwehausen & @amycesal !! 🎉

EOD Update March 31, 2020

MVP for the VA.gov Covid-19 analytics dashboard is up and running with:

• Available Google Analytics metrics
• Available Contact Center metrics
• Available VEText metrics

Dependencies/Blockers

• Please note that Contact Center & VEText data is manual and does not automatically update daily
• FAQ entraces is a screenshot due to Domo quota limits. We hit the quota limit with this metric, which would be unblocked by BigQuery through higher quota limits.

Next

• Looking at additional Contact Center data & Medical hub data. This would involve additional data wrangling, but we will also look for stakeholder feedback on next steps.

---

@KevinMHoffmanUSDS @leahbannon @clantosswett @claricechan

[joanneesteban]

@KevinMHoffmanUSDS

New Blocker

• The team noted that 2 of the charts that are supposed to pull data daily and automatically from GA (charts: FAQ Page top 10 links clicked and Coronavirus Banner on VA.gov homepage clicks) were not successful in automatically updating this morning and had to be fixed manually by @jonwehausen today. This was due to hitting the Domo quota limit, which we would not expect with BigQuery.

Updated Current Blockers

• 2 charts sourced from Google Analytics hit the quota limit when they were supposed to be automatically blocked. We had to manually fix this. This would be unblocked by BigQuery through higher quota limits.
• Contact Center & VEText data is manual and does not automatically update daily
• FAQ entrances is a screenshot due to Domo quota limits. We hit the quota limit with this metric, which would be unblocked by BigQuery through higher quota limits.

[joanneesteban]

MVP - V1 "Released" EOD Tuesday, March 31

Next

@jonwehausen @amycesal for tracking work, do y'all want to just keep this issue open? Or start a new issue for cleanliness?

User Feedback for Next

• [ ] Include all of March for all the charts (or at least back to the first vetext blast) - currently, this is VCL and MyVA311
• [x] For the A/B/C date indicators, show vettext blast dates on all of them
• [ ] Is it possible to rank the orders of the keys from most to fewest? 🤔
• [x] For FAQ Page top 10 - is it possible to make this graph a little larger and show all of the key items without "more..."? If not, suggested content edits: "Centers for Disease Control and Prevention (CDC)" -> "CDC" and "testing criteria provided by the CDC" -> "CDC testing criteria" and "send your doctor a secure message" -> "send a secure message"
• [x] For Google Analytics screenshots, what is the status on integration? If integration is not possible, can we download and update csvs weekly?

Additional Research

Looking at additional Contact Center data, Facility Locator, & Medical hub data. This would involve additional data wrangling.

[amycesal]

I added the VEText dates on every chart that it makes sense to add them. I also changed the size of the FAQ chart so everything shows.

I sorted the items for the keys, but it still doesn't seem like they are showing most > fewest. @jonwehausen or @nedierecel some help here would be appreciated.

We are still missing data for the second VEText. Also including what the text said for the first message would be helpful for context.

[joanneesteban]

Follow Up

@amycesal @jonwehausen

More Immediate

• How far are we with replacing the FAQ screenshots? If the Google Sheets workaround is successful, will this need to be manually updated or is it automatic?
• Second VEText

  Dept of Veterans Affairs COVID-19 update: Stay home, stay safe, stay connected. VA has online tools for your appointments, prescriptions, and more. http://go.usa.gov/xdJkp Send STOP to end messages

Next

• Add in all of March for the FAQ and banner charts for consistency. Since we don't have data for before launch, if we can't add in blank data for the beginning of March, let's add in an annotation that that's when the FAQ and Banner was added
• Top 10 items in order, or most to fewest from Amy's comment - is this something we can implement?
• For the manual updates to make sure we're updating the dashboard on a regular cadence, let's pick a day/time where we QA and update the CSVs. My vote is for Monday mornings, but let's chat so we're can coordinate on this one.

[joanneesteban]

Annie Chatbot metrics have been added and "ready for visualization." Leah's looking to see if she can get more context around these.

[jonwehausen]

Follow-up 4/8/2020

@joanneesteban @amycesal

We now have a Source / Medium visual (connected to GA directly 🎆) living on the COVID-19 dashboard. This visual represents source / medium traffic (entrance metadata) directly to www.va.gov/coronavirus-veteran-frequently-asked-questions/

The dataset supporting this visual is set to update every morning.

Each of the Navigational datasets (supporting the homepage banner clicks and COVID FAQ page clicks) have since been adjusted to push back to March 1, but keep in mind data was not yet available for most of these things as they were not launched or our functionality was not yet built. An annotation has been added for the FAQ page at March 18 indicating launch.

[joanneesteban]

Meeting Notes from 4/9/20

Agenda

Immediate

• When to update the manual stuff on the dashboard
• How to add new things (like CSVs) low-fidelity
• How to visualize these new things
• Questions for Lauren
• Jen Lee’s implementation
• Add annotations for when implementation on FAQ & banner started
• 3rd VEText supposed to go out tomorrow (Friday)

Next

• Visualize FAQ rankings
• Get data from second & third VEText
• What additional medical hub data can we add

Technical Blockers or Dependencies Questions, concerns - where do we want to go from here?

Notes

For manual updates:

• If there’s a need to report on CSVs sooners in the process, we may need a coaching session for everyone. Convert everything to Webforms. And coach everyone on how to add data (record a session).

Questions for Research:

• What action item is the most critical at this time?
• What would be a fire alarm for her (if she saw something spike or dip)?
• What are you hoping to find?
• What is your hypothesis that you’re hoping the data will support?
• What timeframes are you looking to analyze (e.g. before vs. after)?
• What area of the VA are you hoping to assess? Within that, what areas of these services are they hoping to focus?
• How frequently do you need this data?
• Who is the audience for this data?

New Data

• Mental Health
• FAQ and scroll tracking
• Products Used
• Honing in on conversions
• Joining datasets?

Other Stuff:

• Add annotations or synthesis. Maybe once a week as a part of our manual process, we synthesize findings at the top of the dashboard.
• Set Expectation: some of the GA things (like telehealth) aren’t tracked within our properties.

Right now:

• Joanne set up Monday weeklies
• Jon & Brian prioritize Covid implementation

[joanneesteban]

Dashboard Update 4/9/20

• Annie Chatbot Data updated. Re-formatted the data from CSV to Webform so we can start adding line items to this dataset.
• VAOS Monthly Usage also added as a Webform.

Waiting On The following [emails] have PDF dashboards sent. Waiting on a better format before adding to the dashboard:

• FW: My VA311 COVID-19 Metrics
• FW: [EXTERNAL] VISN 23 - March Dashboard
• FW: COVID-19 Veteran-Focused Content Update: April 7, 2020

[joanneesteban]

Notes from Feedback Meeting 4/10/20

Now Steps for Insights

• Perform weekly manual updates & "executive summary" synthesis on Mondays. If possible, Fridays.
• Visualize the current WIP tables
• Add "table of contents" - users can click to different sections of the dashboard from the top view
• Write Up for how BigQuery can help this dashboard progress

Next

• How might we surface the top services used on VA.gov?
• How might we correlate call center data to other services?
• Follow up with ForeSee questions
• Follow up with Joy for help on visualizations
• Follow up with How-To for adding data to Webforms
• How might we scope down the dashboard?
• Search Terms & Links

Feedback

Top Needs

• Metrics
  • Themes from VCL
  • Where people are clicking for Covid information
  • Veteran traffic
  • VEText info
  • Topics; volumes of calls; covid related questions
• Prioritized Metric: Call Center (is the top of the veteran funnel)
  • Where are veterans reaching out for help?
    • Is there a spike or uptick in Covid calls? Sentiment?
    • Is there higher traffic to specific health content?
    • Are users asking specific prescription questions?

Actions from the Data

• Decisions around themes or topics that come up
• "Where do we need to intervene?" and "Are our interventions working?"
• Intervention examples: Blogs, Texts, Content to address user needs

Would be helpful

• To add the ForeSee questions that are under PRA approval: "Did this answer your question"
• Social media & sentiment

Call Center visualization

• Need to figure out how to correlate to other data

Timing

• Lauren sends reports weekly to the White House & Congress on Friday evenings. Begins pulling together the report Friday mornings.

[joanneesteban]

Notes from Charles

• Would it be possible to add weekly or daily traffic count to the main Covid pages?
  • Public Health or the Mental Health pages?
• Looking for an easy summary for the traffic to the main Covid pages

---

Insights: Need to talk to Amy & Joy about what an executive summary and/or rollup chart.

[joanneesteban]

Synthesis for 4/17/20

• Look at VETexts - Users: "do we need to change how we're sending these?" "Could VEText be used more strategically for one-time texts? Could it be used to target more geographic locations, etc.?"
  • Mental Health Click Throughs

[joanneesteban]

@jonwehausen @bsmartin-ep @amycesal @Kimberley2019 @clantosswett

Manual Updates

We should provide the following updates as soon as possible:

Change all manual charts to Webforms

• @joanneesteban I'll do this before tomorrow.
• I'll also change the WeekNums on some of the charts to dates

Visualize the following Call Center tables & add to appropriate sections

• MHV Response VHA COVID Comms - Rx Refill Request by Week
• MHV Response VHA COVID Comms - SM Sent by Week
• MHV Response VHA COVID Comms - 3c- Appointment requests
• Annie Chatbot Covid - Marketing Results
• COVID FAQ More Information Clicks
• FAQ Page Clicks Grouped By FAQ
• VAOS Usage Monthly Report - COVID
• COVID FAQ Page Scroll Depth
• COVID FAQ Helpful Tools Clicks
• MentalHealth VetText Click Thru Rate (CTR) [4/10/2020]
• Mental Health Bounce Rate

Auto-Updates

The following chart wasn't automatically updated over the weekend:

• COVID FAQ Page Entrance Data [Source/Medium] - /coronavirus-veteran-frequently-asked-questions

[leahbannon]

Some adjustments, open to feedback/questions:

• [x] number the table of contents and headers
• [x] adjust title of VCL section to "Veterans Crisis Line (VCL) - Total number of calls vs. Covid-related calls"
• [x] My VA311 has been renamed to just VA311
• [x] Adjust VA311 title to something like "VA311 Interactive Voice Response (IVR) number of selected prompt types per day"
• [x] HRC: adjust title to "Health Resource Center (HRC) total number of covid-related calls per day"
• [ ] for "page entrance data" section on FAQ, adjust to "VA.gov coronavirus FAQ page – top 10 referrers per day"
• [x] for "top 10 links clicked" adjust to "VA.gov coronavirus FAQ page – top 10 clicked links"
• [x] is it possible to label the y-axes on more charts?

[leahbannon2]

For Annie data -- the "precautions" protocol is on there twice. The second protocol in the data that was sent is "Quarantine" and needs to replace the 2nd "precautions" protocol.

[joanneesteban]

We also need a better process for exporting the dashboard, to mitigate:

• single points of failure for exports, particularly if we need to export the dashboard ad hoc
• the amount of manual effort, particularly as we're already above 50 slides

Other Adjustments:

• "Data Added" [date] is inconsistent between charts added automatically and manually
• Section headers aren't exporting, causing confusion on which charts are which
• When the charts are exported, each slide should have context as to which section they belong to so users don't have to flip through the PPT
• Text content cards are not exported
• Text content cards that are notes are not exported in the same slide next to the charts they are explaining
• Screenshots are not exported in the same slide next to the charts they are explaining

[joanneesteban]

^also, have we considered if the plug-in will work on CAG?

[amycesal]

Maybe color is all we need to help define categories for the Annie data. @leahbannon let me know if that's enough, or of we need to stack the precautions bars.

Also @leahbannon let me know when you have the definitions for precautions and quarantine messages.

[leahbannon]

• I think that looks fine without stacking
• Annie has other protocols and numbers, so to avoid conveying this is the totality of their numbers, can you change the "Protocol" in the title to something like "Coronavirus-related Protocols"

For the definitions:

• Quarantine: Prompts Veterans to reply with health metrics, including temperature, cough, and trouble breathing. The replies trigger additional messages with actions to take or not take. The protocol also sends tips for personal and mental health and wellness while isolated or quarantined.
• Precautions: Assists Veterans in monitoring for signs of COVID-19 and when to seek care along with education or precautions to take.

[leahbannon]

I'm also realizing I'm switching around in the above between COVID-19 and Coronavirus... have we settled on a preferred term?

[amycesal]

^ made those update for Annie

---

Covid vs. COVID-19 vs. Coronavirus

We have tried to pull in whatever version the data source uses. And VA seems to use both. Examples:

• COVID-19 chatbot
• VA.gov Coronavirus FAQ page and alert banner
• publichealth.va.gov Novel Coronavirus (COVID-19)
• VEText # 1: Dept of Veterans Affairs Coronavirus update
• VEText # 2: Dept of Veterans Affairs COVID-19 update

[joanneesteban]

@amycesal @clantosswett

Another piece for our exports is:

Can we export the dashboards with labels?

Pain Point: Right now, our exported charts are displaying charts that show overall growth, but our stakeholders don't necessarily see the actual numbers and those changes over time.

How might we be able to export charts with labels?

This might end up being another issue we bring up to Domo in our sync with them on Friday.

[joanneesteban]

Now

• Groom the data for 'date' data type to improve filtering.

[joanneesteban]

No major changes should occur on a consistent basis. Any additional changes can be opened in a new issue. Closing this ticket.