search

hackforla / website

Hack for LA's website
https://www.hackforla.org
GNU General Public License v2.0
294 stars 723 forks source link

Website User Behavior Modeling - current visitor data gathering #128

Open ExperimentsInHonesty opened 5 years ago

ExperimentsInHonesty commented 5 years ago

Dependencies

Overview

We need to model the behavior of people on the current hfla website.

Action Items

Modeling Questions

Purpose: To determine information value ranking. What % of visitors click a top menu item? Of that %, what is the breakdown of which menu item they click?

Purpose: to determine if the project's links are of interest to visitors. For visitors who go to the project section (via click or scrolling), what percent click on the project's links? Of that %, how many people click on more than one?

Purpose: to determine if visitors are seeing that the projects are being worked on one of two locations and pre selecting project exploration based on that. Does there seem to be a pattern of people clicking projects that are in one of the two locations?

Purpose: to determine how people are using the maps, if at all. What is the interaction with the link to maps? (Specifically, do people click on the link?) If yes, do they click on something else, how soon do they click on something else? If yes, is it the RSVP via meetup?

thekaveman commented 5 years ago

IMHO any click/etc. tracking should have an explicit purpose or hypothesis. E.g. we could theoretically have click-tracking on every link on the site, but why? Are there specific questions we're hoping to answer by having more granular use data? If not, I suggest we hold off on this until those questions are formulated.

ExperimentsInHonesty commented 5 years ago

@thekaveman I agree. I have updated the issue.

stu562 commented 5 years ago

I was able to take a look at fullstory, simple sign up and got it to work on my local machine, it only needs a script added to the head of the main html file. It tracks all users session into the signed up account.

For privacy concerns, adding css selectors to exclusion list from FullStory user's account dashboard is possible.

switchzer0 commented 5 years ago

I think a way to focus what we're optimizing around would be on generating more of the two critical inputs to H4LA output (skills, project ideas):

Only after the site is set up for the above do I think we get tactical about e.g., button rearrangement. The volume of responses from the 43 survey responses to date supports this approach (given that skills can only be matched to projects if projects of interest exist).

Response Frequency %
Volunteer opportunities 32 74%
Current project information 25 58%
When and where do we meet? 20 47%
Hack for LA leadership / governance structure 7 16%
How to start a new project 5 12%
Sponsors and sponsorship information 1 2%

(data edited/updated 7/2)

gitklu commented 4 years ago

Current progress and low-fidelity wireframes for website re-design: Hack for LA Website Redesign

(Doc includes link to survey results of website feedback.)

ExperimentsInHonesty commented 4 years ago

See me for fullstory user/pass

ExperimentsInHonesty commented 4 years ago

decided to setup a project specific email. Will organize with Wes

ExperimentsInHonesty commented 4 years ago

@harishlingam to get login sorted.

ExperimentsInHonesty commented 4 years ago

@harishlingam please update with:

  1. Progress
  2. Blocks
  3. Availability
  4. ETA
harishlingam commented 4 years ago
  1. Progress - Reviewing collected data. Reviewing FullStory best practices, appropriate segment builds, and event funnels.
  2. Blocks - None
  3. Availability - Mon, Thurs, Fri, Sun
  4. ETA - 2/16
harishlingam commented 4 years ago

A quick overview of FullStory basics

FullStory Fields

FullStory provides the following data for any Segment: Usage trends over time, Segment Health (events per session, session length, and active time), Click type (Rage, Error, Dead), Device/Browser/Screen Resolution Breakdown, Website Referrers.

Page Insights

Within each Session Playback, we can view Page Insights. Page Insights include (1) Click Maps for each element of the Page and (2) Inspect Mode, which shows insights by clicking on the specific page element of interest.

An Engagement Heatmap overlay for each Click Map may be toggled on/off.

More information here: https://help.fullstory.com/hc/en-us/articles/360020624074-Page-Insights-User-Guide

Frustration Signals

Know the difference between Rage Clicks, Error Clicks, and Dead Clicks: https://help.fullstory.com/hc/en-us/articles/360020624154-Rage-Clicks-Error-Clicks-Dead-Clicks-and-Thrashed-Cursor-Frustration-Signals

Frustrated Sessions: https://help.fullstory.com/hc/en-us/articles/360020828013-Frustrated-Sessions

How FullStory attributes clicks:

From FullStory: “We currently use a combination of inferred page and intelligently processed CSS selector to identify and attribute clicks. Sometimes a given CSS selector can match multiple elements in the page. We currently attribute clicks to the first element we encounter. This generally isn’t a problem if the element or it’s parents have sufficient detail with href’s, class names or IDs in the HTML. That being said, we can do better and have a solution in the works to properly attribute clicks to ambiguous elements like this.”

Further Reading

https://help.fullstory.com/hc/en-us/categories/360001614373-User-Guides

harishlingam commented 4 years ago

Current Insights

@cnk @ExperimentsInHonesty The overwhelming percentage of rage clicks are on project cards of the home page (these do not currently link anywhere). Consequently, rage click data will coincide with dead click data.

Heavy usage by our Website Team may be biasing data.

Event Funnels

We want to think about event funnels that may be useful to us. These can be built by Click Event or Page Visit. Possibilities include:

Custom Segments can be saved based on the desired Event Funnel.

There is quite a bit more that we can do with FullStory, but I think the above is a good place to start.

harishlingam commented 4 years ago
  1. Progress - 1. (i) Reviewed collected data and FullStory literature (ii) Drafted overview page of how FullStory works (provided above, as well as added to wiki under Data Scientist section) (iii) Updated Excel workbook based on revised survey forms. 2. Next steps: establish metrics of interest, appropriate Event Funnels, and corresponding Segment builds.
  2. Blocks - Saw notification on FullStory dashboard that we're approaching the limit of our free plan. Need to investigate whether FullStory provides free-of-cost or discount subscriptions for non-profits.
  3. Availability - Mon, Thurs, Fri, Sun
  4. ETA - 2/18
joelparkerhenderson commented 4 years ago

The overwhelming percentage of rage clicks are on project cards of the home page

Great info thanks for summarizing and sharing! Your data matches my testing of real world users.

What needs to happen ASAP to get the project card for gender-equity pointing to a URL? My ideal URL is https://hackforla.org/gender-equity

harishlingam commented 4 years ago

Update: Inquiry sent 2/17 to FullStory regarding free subscription plan for non-profits. Awaiting reply.

harishlingam commented 4 years ago
  1. Progress - No response from FullStory team. Sent follow-up message requesting contact with team.
  2. Blocks - N/a
  3. Availability - Sun, Mon
  4. ETA - 3/19
harishlingam commented 4 years ago
  1. Progress - @harishlingam To download recorded data and create appropriate Segments and Sessions for analytics.
  2. Blocks - None
  3. Availability - Mon, Sat, Sun
  4. ETA - 3/30/20
harishlingam commented 4 years ago

Update: Data export requires a paid plan. We may have to shift to or supplement with Google Analytics or another free-of-cost option.

harishlingam commented 4 years ago

Progress - A folder was created in the shared drive to house site analytics that will be captured on a weekly basis: https://drive.google.com/drive/folders/1QExQjXkimcKnf2VCosxiUB41NL7El4i9?usp=sharing. New Segments and Event Funnels will be created. Blocks - None Availability - Saturday, Sunday, Monday ETA - 4/6/20

harishlingam commented 4 years ago
  1. Progress - The HFLA Full Story account has reached its recording limit. New analytics tools to be setup. We are looking at Google Analytics and Mixpanel. To reach out to @KianBadie to insert code snippets for tools.
  2. Blocks - None
  3. Availability - Saturday, Sunday, Monday
  4. ETA - 4/18/20
ExperimentsInHonesty commented 4 years ago

@harishlingam @ExperimentsInHonesty discussed and decided at this time to extend our testing to HotJar. @harishlingam will put the code that needs to be added to the website in a new issue. This issue will remain open for continuing the decision record.

ExperimentsInHonesty commented 4 years ago

New dependance: Adding this issue #432 to prioritized backlog.

ExperimentsInHonesty commented 4 years ago

This issue will reviewed again when we have Google Analytics setup #503 . And then also review again the recent survey responses about what people think of the website. We should do a specific report, that is all the surveys that have questions about the website, into one reporting tool that the website team can look at the weekly team meeting.

Here is a spreadsheet that contains all the survey responses. https://docs.google.com/spreadsheets/d/1hvf8By61_7-DWViYFvKKuey1tdYLR-9nc7WCWjyu1dY/edit?usp=sharing

ExperimentsInHonesty commented 2 years ago

@Sihemgourou This issue is in the icebox because of a dependency (#503) which is now closed. Also I added this note the last time we looked at this. Can you review and see if this can be put in the prioritized backlog