Ensure the Info Notice is only shown when suitable data is available

[techanvil]

Feature Description

Ensure that the Info Notice is only shown when there is suitable data available (e.g. the metric which the message is discussing is in a non-zero state) for the current message.

If there is no suitable data for the current message, the next message can be shown instead, with the current one shown when data does become available.

See info notice in the design doc.

---

Do not alter or remove anything below. The following sections will be managed by moderators only.

Acceptance criteria

• The Info Notice should only be shown when there is suitable data available for the current message.
• If there is no suitable data for the current message, the next message can be shown instead, with the current one shown when data does become available.
• In order to avoid a layout shift, the notice may be shown on the page load subsequent to determining the conditions for showing the message, in order for the requisite data to be cached and presented to the client on page load. It might however be turn out to be unfeasible to avoid a layout shift in all cases so this isn't a strict requirement.
• The specific per-message conditions are as follows, with each message listed in turn:
• 1.
  • Message: The higher the portion of new visitors you have, the more your audience is growing. Looking at what content brings them to your site may give you insights on how to reach even more people.
  • Conditions: The "new visitors" audience should be selected and have a non-zero user count over the past 90 days.
• 2.
  • Message: Select up to three visitor groups to display on the dashboard and easily compare metrics between them.
  • Conditions: There should be less than three audiences in the current selection, and at least one unselected audience in the list of available audiences.
• 3.
  • Message: Configure your own custom audiences in Analytics to gain deeper insights into visitor behavior. For example, consider creating an “Existing customers” or “Subscribers” group, depending on what goals you have for your site.
  • Conditions: This should not be shown if the user has already created both an audience named "Existing customers" and an audience named "Subscribers".
  • Note: The initially implemented copy for this message is slightly different, and it should be updated to match the above. See https://github.com/google/site-kit-wp/issues/8140#issuecomment-2179087687.
• 4.
  • Message: Select the "Purchasers" visitor group to gain insights into which visitors bring the most revenue to your site.
  • Conditions: The Purchasers default audience should not be selected, and should have a non-zero user count over the past 90 days.
  • Note: Again, the copy should be updated to match the above.
• 5.
  • Message: The more returning visitors your site has, the stronger and more loyal an audience you’re building. Check which content brings people back to your site - it might help you create a strategy to build a community.
  • Conditions: The "returning visitors" audience should be selected and have a non-zero user count over the past 90 days.
• 6.
  • Message: Compare the ratio of “new” to “returning” visitors – this can give you insights on whether you have more people stopping by as a one-off, or more loyal visitors.
  • Conditions: Both the "new visitors" and "returning visitors" audiences should be selected and have a non-zero user count over the past 90 days.
• 7.
  • Message: Check the cities which bring you more new vs more returning visitors – there might be new audiences you could engage with in locations you hadn’t thought about.
  • Conditions: Both the "new visitors" and "returning visitors" audiences should be selected and have a non-zero user count over the past 90 days.

Implementation Brief

• [ ]

Test Coverage

QA Brief

Changelog entry

[techanvil]

Hi @aaemnnosttv, please can you review the AC for this one? This fleshes out requirements that we didn't get into the detail of in the design doc that would be useful to get a second opinion on.

[aaemnnosttv]

Thanks @techanvil and apologies for the delay getting back to this one. I'm guessing some of the criteria may have become a bit stale since initially added but I've added some thoughts for you below.

The "new visitors" audience should exist and have a non-zero user count over the past 90 days.

Should this be simply non-zero regardless of the date range? IIRC we aren't doing special checks for 90 days any more, correct? Several of the messages use this as well.

Configure your own custom audiences in Analytics to gain deeper insights into visitor behavior, for example consider creating a “Existing customers” or “Subscribers” segment, depending on what goals you have for your site.

Suggestion

- Configure your own custom audiences in Analytics to gain deeper insights into visitor behavior, for example consider creating a “Existing customers” or “Subscribers” segment, depending on what goals you have for your site.
+ Configure your own custom audiences in Analytics to gain deeper insights into visitor behavior. For example, consider creating an “Existing customers” or “Subscribers” group, depending on what goals you have for your site.

Also, should we be saying "Audiences" since we call them "Visitor Groups" in SK? It's potentially confusing to not use the same terminology in both places but I know this was discussed/decided early on. Do we say anywhere in SK that VG are essentially GA Audiences?

Regarding the conditions, several of them reference audience names. Should we add a point to clarify that the actual identification of "matching" audiences should use the respective token in its immutable definition?

Select the Purchasers visitor group to gain insights into which visitors bring the most revenue to your site.

Other messages quote the name, so we should be consistent about how we treat these across all messages.

Compare the ratio of “new” to “returning” visitors – this can give you insights on whether you have more people stopping by as a one-off, or more loyal visitors.

Do we surface the ratio anywhere or do we simply mean to compare these numbers. Maybe we can simplify this a bit?

Check the cities which bring you more new vs more returning visitors – there might be new audiences you could engage with in locations you hadn’t thought about.

The conditions for this one are about the existence of new and returning groups. Should we maybe modify to be specific to these being selected rather than simply existing?

[techanvil]

Hi @aaemnnosttv, thanks for the feedback.

The "new visitors" audience should exist and have a non-zero user count over the past 90 days.

Should this be simply non-zero regardless of the date range? IIRC we aren't doing special checks for 90 days any more, correct? Several of the messages use this as well.

Hmm, my thinking here was that by doing a 90 day check, the user at least has data for one of Site Kit's date ranges if not more. It would also avoid the messages potentially showing/hiding with a layout shift when switching to another date range (this is something that could happen if we check for data for the current date range).

If we only check if the audience has ever had data, we could end up showing the message even though there's no visible data for any of the selectable date ranges, which is a bit counter to the spirit of Mariya's original requirement here, which IIRC is that the messages should make sense in the context of the visible metrics.

Maybe I'm giving too much weight to the layout shift aspect and we should in fact check if there's data for the current date range, as this would be closer to the core requirement here. WDYT?

Suggestion

- Configure your own custom audiences in Analytics to gain deeper insights into visitor behavior, for example consider creating a “Existing customers” or “Subscribers” segment, depending on what goals you have for your site.
+ Configure your own custom audiences in Analytics to gain deeper insights into visitor behavior. For example, consider creating an “Existing customers” or “Subscribers” group, depending on what goals you have for your site.

SGTM, I've added a note to the AC update the copy.

Also, should we be saying "Audiences" since we call them "Visitor Groups" in SK? It's potentially confusing to not use the same terminology in both places but I know this was discussed/decided early on. Do we say anywhere in SK that VG are essentially GA Audiences?

We don't currently have plans to mention it in the plugin. It's certainly something that we should cover in the support page(s). I agree it's confusing!

Maybe we should update this message to say "visitor groups" instead of "audiences", but include a "Learn more" link that takes us to an appropriate section of the support page...

Regarding the conditions, several of them reference audience names. Should we add a point to clarify that the actual identification of "matching" audiences should use the respective token in its immutable definition?

I don't think we need to; the conditions aren't user-facing and as a dev team we know what "new visitors" and "returning visitors" are referring to without needing to expand on it in the AC. They are referred to in a similar way in lots of other issues.

Select the Purchasers visitor group to gain insights into which visitors bring the most revenue to your site.

Other messages quote the name, so we should be consistent about how we treat these across all messages.

Good point, I've added a note about this too.

Compare the ratio of “new” to “returning” visitors – this can give you insights on whether you have more people stopping by as a one-off, or more loyal visitors.

Do we surface the ratio anywhere or do we simply mean to compare these numbers. Maybe we can simplify this a bit?

We don't surface the ratio, no. How about simply replacing the word "ratio" with "number"?

Check the cities which bring you more new vs more returning visitors – there might be new audiences you could engage with in locations you hadn’t thought about.

The conditions for this one are about the existence of new and returning groups. Should we maybe modify to be specific to these being selected rather than simply existing?

Makes sense. That being the case, we should probably update the the conditions for all of the messages which depend on the existing of new/returning visitors to instead depend on them being selected. I have updated the AC for messages 1, 5, 6 and 7 accordingly, please see what you think.

One last point - it's worth noting all the original message copy is taken from the design doc, I added a few to get started, and Mariya tweaked these and added the rest. So please bear that in mind when considering changing the messaging. I'm happy to go ahead with the changes as discussed (pending any further review iteration), assuming we do I'll add a note to the "changes during engineering" section in the DD.

[techanvil]

Moved to Backlog until we get all the P1 issues done and/or the initial release out.