How will we handle recommendations?

[jasonadamyoung]

Following is a list of items (which may not capture all of the decisions needed) that will need decisions to proceed with an initial recommender system implementation:

Note: this list is still in draft form

What are means of conveyance for recommendations?

We've focused on email, and there's a port of the knappsack design (viewable in demo) already. Are we going to display recommendations in any other places?

What timing governs delivery of email recommendations?

We've said 'weekly' previously. But what does that mean? Do recommendations go out on a particular day? Are they spread throughout the week? This goes with...

What content is in an email recommendation?

The first obvious choice is "events occurring next week" - but that assumes a late-week delivery. One could do events "through next week" - which is Time.now <= Event#session_start <= (Time.now + 7 days).end_of_week (and that's what I've checked in to this point).

But are there other events that should be in recommendations? Events that are past but have a recording added? How are those mixed in? Are they separated in display? How many of each do we pick?

What related issues do the emails present?

• How do we handle "unsubscribes"? (dependent partially on Issue #23)
• How do we handle replies and bounces to the "From:" address? (and what technical implementation is needed to support that?)
• What measures do we have in place to prevent us from looking like a spammer?
• How will emails be queued for delivery?
• Are we going to track clicks on the links in the emails to sessions? (If so, we need a per-learner coded link to log the click)
• Are we going to bug the html email to get a sense of whether the email was opened?

How will we make recommendations?

Issue #28 defines an "activity-based" recommender system. But that only has relevance when a person has activity (see EventActivity model for types of activity). For those that have not used Learn - how will we send those persons recommendations? We've talked about "tags" and "AaE activity" - but where do we get that data? Issues #29 and #30 need decisions in relation to any data we might pull from darmok.

[benmacneill]

What are means of conveyance for recommendations?

• I think we have to offer a webpage containing the same info because email client rendering will fail for some people. I don't have anything else on my radar (i.e. tweets, facebook posts, etc)
• I was thinking it was a monday morning email
• Yes, I think we should track links using GA variables
• Yes, we bug to see if the email was opened

[jasonadamyoung]

Summarizing decision from campfire discussion:

• Recommendations go out on Monday morning at 4am in the person's timezone for sessions that will go through the following Sunday (the ISO 8601 end of week).
• We will provide recommendations for upcoming and past events both. Provided that the learner has not already made a connection with the event.
• We'll track links in-app we may use GA as well, but we'll track links in-app so we have the data that we can use for determining recommendation response rates at the learner level.

[jasonadamyoung]

Additional decisions:

• Unsubscribes will be a "single-click" unsubscribe whether the learner is signed in or not
• We'll bug the email.
• We'll have an exact duplicate of the sent email rendered to some location for the purposes of "view in a browser" - for now, we'll keep them for at least a week, maybe longer. "view in a browser" links in old emails need to trigger a friendly error message.
• Will create a related TBD issue for a more general recommendation page in-app.
• The policies on handling replies and bounces will be punted to a later date.

[jasonadamyoung]

All items are now decided except for recommendations to be made to those that have no activity within Learn. These may be deferred until TBD as well.

[jasonadamyoung]

Strikethroughs on all issues decided, moving to TBD.

[jasonadamyoung]

Closing. - replies and bounces will be created as a separate issue. Last issue will be explored as part of Issue #74