Data pull for OES messaging experiment

[erniedeeb]

OES is following up on the messaging experiment they ran with the outreach team. With our help, messages were sent to segments of subscribers using the campaigns below. We'd like the pull these numbers despite the analytics issue that ran through the duration of this experiment. We should also include a disclaimer about how that issue affected the numbers in the report. Requested metrics:

• Use of the benefits tool (making it past page one) at the aggregate level for each version of the email (treatment group)
• Activity on the benefits landing page (going to the tool vs. a different page vs. nothing) at the aggregate level
• Making it through the benefits finder to the last page at the aggregate level
• Sharing results at the aggregate level

This report doesn't have to be super detailed (e.g. showing drop off at every step of the funnel).

Requested date range: 5/15 to 5/29 Campaign links:

• Spanish:

• • https://www.usa.gov/es/buscador-beneficios?utm_source=email&utm_medium=email&utm_campaign=long_general
• • https://www.usa.gov/es/buscador-beneficios?utm_source=email&utm_medium=email&utm_campaign=long_tool
• • https://www.usa.gov/es/buscador-beneficios?utm_source=email&utm_medium=email&utm_campaign=short_general
• • https://www.usa.gov/es/buscador-beneficios?utm_source=email&utm_medium=email&utm_campaign=short_tool

• English:

• • https://www.usa.gov/benefit-finder?utm_source=email&utm_medium=email&utm_campaign=long_general
• • https://www.usa.gov/benefit-finder?utm_source=email&utm_medium=email&utm_campaign=long_tool
• • https://www.usa.gov/benefit-finder?utm_source=email&utm_medium=email&utm_campaign=short_general
• • https://www.usa.gov/benefit-finder?utm_source=email&utm_medium=email&utm_campaign=short_tool

[fongcindy]

Ernie has requested to get this data by 8/9.

@nickpistone @EnriqueOrt-GSA

[nickpistone]

https://docs.google.com/spreadsheets/d/19hAeviG5aPJDMQQfzSt7Cxo6h1jD1s-wuCAbQMf55ag/edit?usp=sharing The first tab shows the sessions by campaign and page. This tells us how many people clicked the link(s) in each of the four email segments. The second tab removes the page column so that you can see the totals for each campaign. And the third tab shows the number of sessions that included an accordion open for each email segment. It also shows an adjusted value, because accordion opens were not firing 100% of the time during this period. Based on the previous trend, the value shown is about 66% of the true value. So the adjusted values add back in the other third. Then I calculated the percentage of sessions with an accordion open for each version of the email using the adjusted value. The "long_tool" version of the email had the lowest number of clicks (sessions), but the best engagement as a percentage of sessions.