Feedback: allow readers to thumbs up/down on a page

[humitos]

We've been asked multiple times about the ability to receive feedback from readers on a particular page.

With the new addons client I think this is probably a good idea to consider. We could implement this feature in an agnostic way that works for all the doctools by:

• Showing a floating element with 👍🏼 / 👎🏼
• Hitting a /_/api/v3/feedback/ endpoint on click
• Saving the feedback on a PageFeedback model
• Listing all the PageFeedback analytics under an admin page

Open questions

• How useful this would be be?
• What use-cases this idea would solve?
• Would it be better if this supports commenting on pages besides 👍🏼 / 👎🏼 ?
• How we would display the analytics? Just a number of 👍🏼 / 👎🏼 per page would be enough?
• Would SPAM be a problem here?

(I'd like to collect some feedback here 😄 )

Related issues

• https://github.com/pypa/pip/issues/8784
• https://github.com/readthedocs/readthedocs.org/issues/6
• https://github.com/readthedocs/readthedocs.org/issues/3568
• https://github.com/readthedocs/readthedocs.org/issues/2533

[ericsnekbytes]

At the JupyterLab weekly meeting, we discussed the desire for these types of metrics collection on our docs pages.

For our project, the above proposal for collecting user feedback per page and making those available via an admin panel would be great. Expanding on some of your questions:

• How useful this would be be?

  • Very useful. This would save us from either paying for related services, or building our own solution (neither are appealing because of resource limitations). ReadTheDocs is trusted and respected, so working with you to obtain this information gives us confidence that we wouldn't have with other vendors/solutions.

• What use-cases this idea would solve?

  • Drive docs improvements: If we know which pages people are frustrated by, we can organize additional efforts around the pages most in need of attention
  • Granular item/topic feedback: By allowing this type of feedback not just on the page, but on sections/headings, this could drive topic-level improvements by allowing readers to report dissatisfaction or broken/outdated information with a section or area
  • Qualitative feedback via open text form

• Would it be better if this supports commenting on pages besides 👍🏼 / 👎🏼 ?

  • A thumbs up/down would be valuable, but for Jupyter we discussed some potential additional formats:
  • 5 button scale from "Very Unhelpful" to "Neutral" to "Very Helpful"
  • Open text feedback (maybe an overlay/dialog after users click a feedback button) would give us additional granularity and qualitative information to drive specific, focused improvements to documentation content

• How we would display the analytics? Just a number of 👍🏼 / 👎🏼 per page would be enough?

  • CSVs for download would also be nice to have (similar to page count CSVs per month that we have now)

• Would SPAM be a problem here?

  • If we get spam, we could perhaps filter it out from the data if given the CSVs
  • Making methods available for limiting spam would probably be helpful (perhaps some type of captcha integration for instance).

Thanks for your work on this!

Edit: N-button scales could potentially be enabled via a config file, with associated icons/unicode per item, then the button ID could be captured on click and stored via the admin panel.

[humitos]

Thanks a lot for your feedback! 💯

Summarizing ✍🏼 what your wrote for my future self:

• Export the data via CSV file for better analysis
• Option to configure whether using 👍🏼 / 👎🏼 (good / bad) or ⭐⭐⭐⭐⭐ (gradual score)
• Not only per-page feedback but also more granular: section/heading
• Allow feedback via open text form

[seibert]

For analysis purposes, also having the CSV with a time stamped row for each interaction (up, down, star, etc), rather than pre-aggregated, would allow us to filter during analysis based on when documentation changes or new software releases go live.