Closed floere closed 9 years ago
I want to think a bit about what data we can work with to do this sort of stuff, what are the metrics you use when looking at a dependency when comparing libraries:
Metrics | Reason | Source | Indexed | Shown | Sorted |
---|---|---|---|---|---|
Documentation % | Documentation Quality | CocoaDocs | √ | ? | |
Size of library, e.g. class count | How big of a dependency is it | CocoaDocs | √ | ? | |
Code Coverage % | Code Quality | CocoaDocs | √ | ? | |
README complexity % | Documentation Quality | CocoaDocs | √ | ? | |
CocoaPods install numbers | Popularity | Analytics | √ | ? | |
Github Stars | Popularity | Metrics | √ | √ | |
Github Forks | Popularity | Metrics | √ | √ | |
Github Contributors | Popularity | Metrics | √ | √ | |
Github Open Issues | Participation | Metrics | √ | ? | |
Github Watchers | Popularity | Metrics | √ | √ | |
Screenshots | Documentation Quality | trunk? | √ | ? | |
Social Media | Documentation Quality | trunk? | √ | ? | |
Author | - | trunk | √ | √ | ? |
Dependencies | - | trunk | √ | √ | ? |
Name | - | trunk | √ | √ | √ |
Platform | - | trunk | √ | √ | |
Summary | - | trunk | √ | √ | |
Tags | - | trunk | √ | √ | |
Version | - | trunk | √ | √ |
Datapoint | Source | Indexed | Shown | Sorted |
---|---|---|---|---|
Last Release Date | Github | √ | ||
Number of Files | CocoaDocs | √ | ||
Last Release Date | Github | √ | ||
Most Recent Commit Date | Github | √ | ||
Open PRs | Github | √ | ||
Open Open Issues | Github | √ | ||
Dates of last releases | Github | √ |
( ^ people with admin access, feel free to edit this table )
@orta Awesome!
@kylef added
http://cocoapods.org/?q=kylef would be cool if we could do github usernames
@orta Remember that we spoke to Jonathan Penn and his colleague at ‘WWDC after dark’? I think it’s time to ask him to start thinking about scoring.
@orta Re: "full comprehensive overview" – I wonder if the search engine or cocoapods.org would be the right place to host this. Probably not the search engine, as it's probably best to have it do searching exclusively. Where would pod detail pages be best hosted? Do we need a new mini-service which only renders beautiful HTML of pods?
@floere I have been missing something like this for ages: https://rubygems.org/gems/cocoapods
To me it makes sense to have it in cocoapods.org
because after you see a list of Pods (with the search) you might want to see the detail of a single one.
@irrationalfab That's exactly what @orta is proposing! :)
correct, cocoapods.org will probably have to move to be a sinatra app to do proper routing IMO.
@irrationalfab Since cocoapods.org is a static page, and we do need dynamic content, I was wondering who would provide it.
An example of how to display a list of packages: http://www.staticgen.com
@orta I think would be good to use the (future) pod info API, which provides the info you describe above and basic infos – this would push us to actually implement such a pod info API, for ourselves.
Definitely, whatever we make for this should be an API we dogwood and release for that things like the alternative cocoapods search engine can do it
We should also not forget about this experiment: http://cocoapods.wantedly.com
We should also not forget about this experiment: http://cocoapods.wantedly.com
I like what they did there
@irrationalfab Already in the issue description :)
:sob:
:dancer:
@orta When can you devote some time on this? The rewrite to Heroku is (basically) finished and you could continue. Or we could put it online first, and then you start working. Either way is fine for me.
@AliSoftware adds some thoughts here: https://github.com/CocoaPods/search.cocoapods.org/issues/56#issuecomment-58732227
The only options I think of and that are not listed in Fabio's table yet are:
Metric | Reason |
---|---|
Name matching | We search for a specific pod already |
Last update | We want a maintained pod, not an abandoned one |
Number of versions | We want a maintained pod, not an abandoned one |
I'm on a deadline for another 2 weeks, then it'll be my main CP priority
If you have time to add the sorting before then, it can go in, the user-facing side is all JS + HTML that can be ported over with a C+P
Maybe we can do it in steps? With the first step being an enhanced pod results view/page.
Well the inline results will basically be the same a pod page, so I think that is the best first step:
Great! I've added a checkbox in the description.
@orta Re sorting: what should it be sorted by? Any ideas?
A sort by popularity ( there should be multiple sort types, this is the first ) would look like:
pod.stargazers + (pod.contributors * 90) + (pod.forks * 10)
right now.
With the addition of watchers, this should go to:
pod.stargazers + (pod.contributors * 90) + (pod.watchers * 20) + (pod.forks * 10)
Tested this locally, and it feels pretty good! :+1:
Thanks! I was at first surprised that stargazers are counted so little, but I assume that your reasoning goes that there are far more stargazers than contributors (that is, you try to normalize the values).
Let's just try it live and see how it feels.
The watchers are in – Github calls/called them subscribers, and so did we (https://developer.github.com/changes/2012-9-5-watcher-api/).
I've added popularity, both negative and positive, so you can try it:
http://search.cocoapods.org/api/v1/pods.flat.ids.json?query=name:reactive&sort=-popularity http://search.cocoapods.org/api/v1/pods.flat.ids.json?query=name:reactive&sort=popularity
Have fun!
Yeah - awesome, to get these numbers we picked out 15 random pods and used our heads to rank them, then started looking at average ratios from stargazers to subs/forks/commiters to get some rough ideas on how they connect. Using that we aimed to try and see changes within the medium sized popular pods.
IMO, you should make a small blog post and we should get this as the default sorting algorithm on the site. We can say that it is basically step one towards cocoapods.org v2.
The post could answer:
Note: I'm going to turn the param around so that sort=popularity
is descending, and sort=-popularity
is ascending (sort of "negative popularity").
@orta Heh, what happened to humble opinions? ;) I'll wait for a day or two to see if any errors occur with the new service, then publish a blog post. I'd like to be sure it works.
Sorting options atm are:
name
popularity
-popularity
contributors
-contributors
forks
-forks
stars
-stars
watchers
-watchers
E.g. the popularity
starts high, goes low. The -popularity
does the opposite.
E.g. the popularity starts high, goes low. The -popularity does the opposite.
Yeah makes sense IMO
Spent some time on figuring out code quality weighting:
| 1.0 | Average popularity of libraries for author ( breakdown TBD, grouped likely )
| 1.0 | Documentation %
| 0.6 | Popularity of lib ( grouped also, want to get mode popularity )
| 0.6 | Readme complexity, higher = better
| 0.2 | Has screenshots
Ideally we get some more metrics that mean that we can also add the following:
| 1.5 | Code Coverage
| 0.4 | Faux Pas Count / Warnings when being ran on CocoaDocs
| 0.4 | Has example project
| 0.2 | Has a travis project
Some of these can be added to the cocoadocs dataset pretty easily.
Introducing FauxPas is a nice idea but we should choose the settings wisely and be sure to run it only on the lib code. A pod with a quick&dirty sample project — that does miss a lot of error checks and all — does not necessarily mean a bad pod and that the code of the lib itself is bad. Especially as example projects are just that, example generally not finished up to the little detail, so they might be a lot of them with high FP warning count.
That being said I like the idea!
Note: IDK which FP license you guys got, just be careful it allows sharing analysis results like that to everyone even to those CP users that don't have a FP license. For example we ourselves wanted to integrate FauxPas on our Jenkins CI servers but discovered it requires as many licences as there are people that have the rights to consult the FauxPas analysis results!
Yeah, my thoughts were to count the "number of rules broken" rather than the count of rule infractions, for example:
This is with default settings but there are 79 rules, and 36 of them were broken, but only 2 were reds. Rather than using the 1000 things wrong :)
But yeah, it's a pretty interesting metric, will obv talk to them
I switched it to sort by popularity
by default.
@orta Looks good – I think we should just try it. Next step: add docs trunk table to strata :)
Another sorting idea: hotness ~= popularity / time_since_first_commit
Ideally there will be 5 sorting choices: Quality (default)
, Popularity
, Freshness
, Maturity
, Name
As a side note, while we do some redesign, we should also add some help about search query syntax, like hints about possibilities like AFNatwerking~
for "similar to", or author:mattt|eloy
for pods from one of those two.
I believe it could go as a note under the search field to show one or two advanced queries examples, + a link to a detailed page for advanced search help with all the possibilities.
I've opened design issues for inline search & pods pages, this one can stay metrics / grabbing only
https://github.com/CocoaPods/cocoapods.org/issues/105 https://github.com/CocoaPods/cocoapods.org/issues/104
@orta Perfect, thanks!
On Fri, Nov 28, 2014 at 6:36 PM, Orta notifications@github.com wrote:
I've opened design issues for inline search & pods pages, this one can stay metrics / grabbing only https://github.com/CocoaPods/cocoapods.org/issues/105
https://github.com/CocoaPods/cocoapods.org/issues/104
Reply to this email directly or view it on GitHub: https://github.com/CocoaPods/search.cocoapods.org/issues/51#issuecomment-64916295
I think we can close this issue, as beautiful and comprehensive as it is…
I'm closing this. There are some sorting options remaining, but the sorting options we have are enough, for now (and possibly underused).
Other people have done some great work at http://cocoapods.wantedly.com/.
How would our best search engine we can come up with's UX be? Directly visible tags? Facets? What do we search for? Sorting?
At this point, do we even know how people search and how they would search if they could search in any imaginable way?
Switching on the old @orta signal and pointing it at the clouds over Podham City.
Next steps: