code-corps / code-corps-api

Elixir/Phoenix API for Code Corps.
https://www.codecorps.org
MIT License
234 stars 86 forks source link

Track member approval #1307

Closed iref closed 6 years ago

iref commented 6 years ago

What's in this PR?

These changes should enable conversion funnel analytics using member_id in queries.

References

Progress on #1285

joshsmith commented 6 years ago

@iref this is a useful addition in other ways so I want this PR to get merged as-is, though I think it's not quite a fix for #1285.

I've had some extensive conversations with a support engineer at Mixpanel who has enlightened me that we may want to track a project_n, e.g. project_25, as a distinct id. This means we'd actually rewrite the events as Membership Requested (Project) and Approved Membership (Project) from the perspective of the project. But we would also track a Requested Membership (User) and Membership Approved (User) from the perspective of the user.

We likely need to tweak the semantics here, but does the gist of what I'm saying make sense?

iref commented 6 years ago

If I understand it correctly, we would like to emit two events on membership request/approval. One from user perspective (this one already exists) and one from project perspective. Both events have same properties.

Can we extract project_id from plug connection or resource and set it as segment api user_id when emitting Membership Requested (Project) and Approved Membership (Project) events? I am not that familiar with Segment API, so I am not sure if this is correct way how to do this.

joshsmith commented 6 years ago

I don't think we'd extract if from the conn but from the %ProjectUser{project_id: project_id}.

joshsmith commented 6 years ago

And to clarify @iref, you're right about wanting to emit multiple events.

Without being confusing, here's the message from Mixpanel (modified slightly to fix some errors I spotted):

Thanks for your response! It doesn't hurt to track more properties than fewer in this case. I'd suggest tracking the following:

  • Event: User Requests
    • distinct_id = the user's distinct_id
    • project = project they want to join
  • Event: Project Accepts Request
    • distinct_id = the project's distinct_id
    • project = project they want to join
    • accepter = the user who accepted the request

That way, you could track this for a single user: the user who requested to join and when the user's request was accepted. If you want to track the user accepting that other user's request to join, you could trigger another event to document this and have the distinct_id be the accepter. The caveat to this is that you'd have 2x events for when a user's request is accepted, but you'd be able to capture this in our existing funnel.

In our case the user's distinct_id would be id, as it has been, but the project's distinct_id would be project_n, e.g. project_1. This is necessary because otherwise the analytics tools (like Mixpanel) would confuse 1 for the user with id of 1.

All that make sense?

joshsmith commented 6 years ago

If you want to squash your commits on this PR down to just a single commit, then I'm happy to merge this as-is since this is a good changeset, and we can continue with the bigger change in another PR.

iref commented 6 years ago

Yes, I think I get it now. I'll start to work on new PR that implements suggested changes.