User feedback on packages - for discovering new packages and learning from users

[sfirke]

I don't have something specific in mind, but I'd love to see an easier way to browse and learn about packages. Things I'd be interested in:

• User reviews (incl. what users don't like or wish for, so maintainers can improve things)
• Recommendations based on other packages you use or "liked"
• Standards: is the package...
  • Actively maintained on GitHub?
  • When was the last update of any sort?
  • Unit-tested?
  • Tidyverse-friendly?

Maybe these are defined standards, maybe not and maintainers can just self-tag as "tidyverse-friendly." But I'd prioritize packages that meet these standards when I'm investigating options.

• A statement from the developer as to its vision, who it's recommended for, what it's good for and not?
• Maybe download activity: popularity level and is it trending up or down?

The closest thing I've seen is crantastic but in practice I can't do the above things with it.

There are >10k packages on CRAN and that number is going up fast. So as much as this is a problem now, I think it'll be a bigger one in a year, and then in three years or ten years.

[batpigandme]

I think that, if done, we'd need to be mindful of some of the “wicked problems” inherent in what is, essentially, a form of "market signaling." Obviously the method has some merit (the wisdom of crowds and whatnot), and works in certain "markets," but there are specific prerequisites that kind of have to be in place (e.g. market thickness).

Let me know if my analogy has gone off the rails here, and I'm happy to try to clarify.*

_I'm kind of pseudo-using the terminology from Alvin Roth's Who Gets What — and Why and I could totally be misinterpreting them._

[noamross]

Some time ago I tried an experiment embedding a rating widget right into the HTML of an R help file. @jonocarroll took it farther: https://github.com/noamross/htmlhelp/pull/16. The idea was to rate documentation but the general concept was to move the rating very close to the user experience. I could see something like an RStudio addin that prompts you to rate the packages that you are using and sends info to this service.

[batpigandme]

Good point, @noamross. I hadn't really considered the benefit from the dev's perspective there.

[jonocarroll]

(commenting because mentioned)

@noamross -- that idea became btts but I was uncertain about the dangers of invading users privacy with that original concept (having an identifiable ping every time I use a package? no thanks) so it just connects the user to the source now.

thankr may be more along the lines of what you're after in terms of starring repos. I'm going to guess there's no way to automate watching other people's stars (beyond following them & checking the GH home page) so a scraper of that would be a great start.

I can definitely see value in a recommender service, and would love to see it incorporate download stats in the style of https://hadley.shinyapps.io/cran-downloads/ but less so community-driven comments (it's a great community but has its detractors).

A GH README in theory covers a lot of the points raised but of course there's no minimum requirements. A new badge service could make this work better, though some of these (covr, last updated, and active are already available). devtools already incorporates most of these into the best practice functions, but some expansion could be useful (I have a feeling Jenny Brian has a write-up of this already).

Have a great time, everyone! I'm looking forward to hearing about all the cool things you work on.

[noamross]

@jonocarroll Oh, cool! Nice work with btts. That's an interesting take on @jennybc's issue #41: promoting linki-ness in packages.

[jimhester]

I think the main issue with any of these community driven recommendation systems is not technical, it is getting the critical mass of recommendations to make them useful. I unfortunately don't have a solution to this problem, although perhaps you could have a rateme dialog popup in .onAttach() X% of the time?

[sfirke]

(renaming the issue to reflect the broader possibilities and angles)

[karthik]

There was a project by Chris Bogart at CMU that tried to do something similar. It's called the Scientific software network map

It never made it to CRAN because it phones home to report package usage. The project might be dead now?

A related idea that has been under discussion by the rOpenSci staff is the idea of bespoke task views. So, not just a complete listing of all packages that do x, but more of an expert curated guide on the current state of tools for a particular topic, with some notes pointing out the pros and cons of each tool. Much like the wire cutter guides.

@jeroen and I recently talked that one such task view could come out of the recently completed text workshop.

So it wouldn't replace CRAN's taskviews but offer a different perspective.

[stephlocke]

I think there's a need for something like Awesome lists or user-submitted task views somewhere that people can pull via a package. I currently do this with shell packages consisting of just DESCRIPTIONs with Imports/Depends to cause them to be fetched - not a very elegant solution!

[karthik]

@stephlocke An entirely different project that I'm working on (with help from @jimhester) will help with this. No additional package necessary.

[stephlocke]

Fantastic :)

[sfirke]

Summarizing: so far I don't see a concretely actionable idea for the unconference. Perhaps something that searches packages with badges described in #58 ? To give an idea of what "standards" are met (per my top post above), and combines that with trending downloads and when the last commits to the code were.

In the face of other interesting, more concrete ideas, I'm inclined to table this unless others want to chew on it for a little bit in person.

[gshotwell]

I had an idea a little while ago for something like the Medium highlight tool but to indicate confusion. I think part of the problem with rating documentation is that authors need to know which parts of the document are confusing, something which let you highlight a piece of text and say "I got lost here" would help with communication.