Speaker Feedback: Create a report for feedback stats

[ryelle]

What can we measure?

• How many people submit speaker feedback (compared to how people attended the event).
• The quality of WordCamp talks, based on the star ratings that are submitted.
• What percentage of speakers actually look at their feedback (we can log view counts).
• The quality of the written feedback based on ratings supplied by speakers as well as manual processing.

[iandunn]

I think we should also track stats around moderation, to see how that affects speaker's ability to receive the feedback while it's still effective:

• % of submitted comments that have been moderated
• Average # of days between the event and when the comments were shared with the speaker
• % of comments that were edited by a moderator (if we're giving them this ability)
• % of comments that were rejected by a moderator
• % of comments that were marked as spam

To maintain accurate data, it may be good to prevent permanently trashing comments, and instead setting them to a custom Rejected, Spam Feedback, etc statuses.

[coreymckrill]

• % of feedback marked as "helpful" by the speaker

[ryelle]

From above, I've pulled out this as the data we want per-event:

• Count of feedbacks per session as a percentage of total attendees marked attended
• Average star rating of all talks
• How many speakers look at the feedback page (as a percentage of all speakers)
  • We'll need to track speakers viewing feedback, could be a boolean post meta per speaker?
• Percentage of feedbacks that have been approved out of total
• Percentage of feedbacks that have been spammed out of total
  • We can prevent akismet from deleting spam comments by raising akismet_delete_comment_interval to an arbitrarily large number.
• Percentage of feedbacks that have been marked inappropriate out of total
• Percentage of feedback marked as "helpful" by the speaker

Anything else?

Process-wise… Something similar to Five for the Future, saving a snapshot of this data at some regular interval? Or should it just save once, maybe a month after the event ends? Attendees can only submit feedback for 2 weeks, so that + another 2 weeks for speakers to view the data sounds like a good time to generate a snapshot. We could run this only on central, and have it save a snapshot record per event.

Or should this be more like the generated reports that pull from each site on-demand?

[coreymckrill]

I think the on-demand generated reports make the most sense here. The snapshots are good for 5ftf because it's an ongoing process with no distinct beginning or end. Here, the beginning of the event is a distinctive "start" and 2-4 weeks out is the "end".

Percentage of feedbacks that have been spammed out of total

I'm not sure this particular stat provides a lot of value, and I'm leery of keeping a bunch of worthless data stored in the database just so we can have this stat. Maybe initially it would be helpful so we could see whether the Akismet integration is working effectively. On that note, I do think it would be interesting to track how many feedbacks are manually marked as spam by an organizer as opposed to getting caught automatically. Perhaps compare it to the number being marked as inappropriate, and we could see if organizers are using those two different options as we intend them to be used.

[coreymckrill]

Count of feedbacks per session as a percentage of total attendees marked attended

Since we currently allow a user to submit feedback to the same session multiple times (logged in or otherwise), we might need to take this a step further and say Count of users who left feedbacks per session

[coreymckrill]

Here's an idea for capturing the total spam stat without having to keep it all in the database:

I said earlier that I don't think static snapshots are the correct solution here, but they could be for the spam count. We could hook into the routine that clears out the spam bucket (I presume) and save an option to the database of how many were deleted. Then when a report is run for all of these stats, the spam count would be that saved number plus however many are currently in the bucket that haven't been deleted yet.