search

nextcloud / appstore

:convenience_store: App Store for Nextcloud
https://apps.nextcloud.com
GNU Affero General Public License v3.0
301 stars 136 forks source link

Introduce a rating system #14

Closed oparoz closed 8 years ago

oparoz commented 8 years ago

Companies behind large app store have taken a lot of care to improve their rating system in order to make it easier for users to find the best apps and have per version rating since everybody knows a lot can happen over time. It would be great if Nextcloud could follow their foot steps.

google_rating amazon_rating

Benefit / value

BernhardPosselt commented 8 years ago

My approach to this issue is currently to only let people recommend apps (something like a like button) which leads to the following ratings:

This is the way we actually work. If you want to buy something and you don't know if its any good, you ask someone that you trust or take a look what most people recommend.

I think a star rating is a bad idea because of the following issues:

Why no dislike button? I think 0 stars or dislike buttons really hurt people because we take negative feedback much harder than positive (I think 1 negative rating equals 10 positives or something like that). Since people essentially work on these things in their free time and for free, I think this discourages people more than it helps. That's also a reason why I'd like to avoid stars.

BernhardPosselt commented 8 years ago

As for the current users happy index: This should be done in the comments and not be included in the overall rating. The latest comments will give you an idea if people like it and you can also enhance comments with thumbs up like githubs issue system.

oparoz commented 8 years ago

This is the way we actually work. If you want to buy something and you don't know if its any good, you ask someone that you trust or take a look what most people recommend.

Yes, except that the like button approach is not a replacement for that. People will still do that and get recommendations from forums, friends, social networks, etc. With a like count, you're stuck at a certain point in time. You wouldn't pick a 2014 printed magazine to get today's rating on anything.

So you're basically left with 1-3 stars which is roughly the same. I don't think a "meh its ok" rating helps

I do. 1 star, avoid, 3 stars, try alternative and come back if nothing better is found, 4 star install

It's averaged which leads to very unimportant apps being rated above apps used by hundreds

Which makes perfect sense if an app used to be popular, but is now something rotten which floats at the top

Both systems have a similar problem. As apps evolve, so should the rating. The advantage of using stars is that you have a bit more nuances. Making a review mandatory when adding a rating, gives more context to a specific rating. A like is too easy to get and nobody cares about removing it, unless they're really angry. Thumbs up/down on reviews would help select the best and worse ones to make the context even better.

oparoz commented 8 years ago

Comment as they are currently used in the oC app store are useless. Everybody uses them as support requests. Now if you remove these, you're left with... reviews, which should be tied to a rating.

BernhardPosselt commented 8 years ago

As discussed on IRC (for everyone else) the idea will be to ask users why they comment after hitting the comment link. We can basically ask them "did you encounter an issue/bug" and redirect them to the issue tracker :)

BernhardPosselt commented 8 years ago

I do. 1 star, avoid, 3 stars, try alternative, 4 star install

So basically 1 and 3 stars are the same as 0 stars for you :)

Which makes perfect sense if an app used to be popular, but is now something rotten which floats at the top

yes, the issue is just that 3 five star ratings are averaged higher than 1000 4 and 5 star ratings

oparoz commented 8 years ago

So basically 1 and 3 stars are the same as 0 stars for you :)

I didn't want to make it too long, but I've updated it now.

1 star, avoid, 3 stars, try alternative and come back if nothing better is found, 4 star install

It's not an exact science, but we always try to find the best tool for the job without having to do too much research

yes, the issue is just that 3 five star ratings are averaged higher than 1000 4 and 5 star ratings

Indeed, but people can see that the app is new, has few reviews and that they don't really describe the experience. We also don't expect dozens of Notes app, so I think it shouldn't be difficult to discover what you're looking for.

oparoz commented 8 years ago

Also, as said on IRC, I think both "likes" and "stars" can co-exist. I would find it useful to see that some apps are recommended by people I trust.

BernhardPosselt commented 8 years ago

Another thing that comes to my mind: usually when I'm looking for a library to use, I'm taking a look at the repository and check:

There are some tools and libs that do this which could also use (probably only on the website) for these kinds of things

oparoz commented 8 years ago

Yeah, I do the same thing, but there are issues

When was the last commit

Transifex skews the result, but we can use the last time the app was uploaded to the store

How many bugs are registered in the bug tracker

That's a problem with trackers such as Github, unless you can filter by label and make it mandatory to have a "bug" label

You could also maybe retrieve the number of contributors. Not a perfect indicator, but as repos become popular, new devs try to participate.

BernhardPosselt commented 8 years ago

Yep, it's also very hard to abuse this kind of rating :)

BernhardPosselt commented 8 years ago

Or at least it would be absolutely obvious (think of 50 opened and closed issues in a day xD)

adsworth commented 8 years ago

For me interesting metrics are:

I like the djangopackages metrics quite a lot. auswahl_124

oparoz commented 8 years ago

Number of installations

That would be really good if we could get the number of installations per version, with the default being the latest.

But there is a problem with all these dev stats, it favours popular, older apps. A really good, niche app will not easily be discovered or featured.

BernhardPosselt commented 8 years ago

@oparoz apart from that it can't be enforced since nextcloud downloads the app by itself (we only serve the download link) :)

BernhardPosselt commented 8 years ago

Which is a bonus for me because you know, strange people might set up download scripts and basically DDoS the server xD

oparoz commented 8 years ago

@BernhardPosselt, Well, the API knows exactly which download request is being made, so you could have stats there, no?

BernhardPosselt commented 8 years ago

The API does not serve the archive, no. It serves the full repo as JSON including the download links

oparoz commented 8 years ago

Ah, yes, and showing the link is not the same as downloading the app...

adsworth commented 8 years ago

But there is a problem with all these dev stats, it favours popular, older apps. A really good, niche app will not easily be discovered or featured.

This could be smoothed out by displaying the average installations per month since the app is in the app store and also a trending indicator. i.e. number of installs is on the rise, or number of installs is falling.

We should also track uninstalls. since then the "Number of installations" would be a much more significant metric. The install is actually not the interesting part. If 100 people install it but then 90 deactivate it because they don't like then the "Number of installations" doesn't have much meaning.

Also I think we should talk about activation/deactivation instead of installation.

despens commented 8 years ago

Some nice ideas here. Personally I don't see this as a must-have feature given the very low amount of apps available right now. It is easy to scroll through them, and there are hardly any alternatives to chose from. As soon as there are five calendar and seven mail apps, this would make sense.

oparoz commented 8 years ago

@adsworth - I agree with what you're saying, but unfortunately we can't track activations.

@despens - I think it's useful as soon as there are 2, but overall I agree. No system is better than a bad one like in the ownCloud app store which sends the wrong message.

MorrisJobke commented 8 years ago

I also like the Arch User Repo popularity rating:

Popularity is calculated as the sum of all votes with each vote being weighted with a factor of 0.98 per day since its creation.

MorrisJobke commented 8 years ago

And I second @BernhardPosselt comment about likes. I would also go for a simpler metric. Combined with the above mentioned popularity it would make easy and doesn't favor the old apps anymore.

jospoortvliet commented 8 years ago

Agreeing with Morris. With regards to star ratings, I have to share this here...

I do agree that this is something we can work out later as, for now, it isn't that there are 10000 apps.

oparoz commented 8 years ago

Steam has a good model.

Temporality

steam_overall

Simple rating

steam_good_bad

Trustworthiness

steam_trust

BernhardPosselt commented 8 years ago

Oh very good point. I love Steam's rating system especially now that they included the recent and overall rating and also giving a comment a rating makes a lot of sense.

So basically in addition to the current rating data model we need to record the time when it was rated and also add a boolean flag for likes/dislikes, e.g.

class Rating(Model):
    user = ForeignKey(User)
    app = ForeignKey(App)
    positive = BooleanField()
    rated_at = DateTimeField(auto_now=True)

You can then basically do something like youtubes progressbar (red and green). Limit the likes and dislikes to 3 months and you'll get a recent score. You can also way in the comment ratings, however that would be an advanced topic. I like the idea.

Two qestions are popping up in my head however:

PS: we should also cache the rating on the app model itself since its probably very time consuming to calculate it.

BernhardPosselt commented 8 years ago

Moving this to 0.1.0

If we decide to use a different rating system than simply likes we should get rid of the API recommendations field so we don't break the API later on.

Implementation can follow later however.

BernhardPosselt commented 8 years ago

Closed in favor of #212 , #211 , #210 , #209