Implement stemming of words

CocoaPods / search.cocoapods.org

This is the CocoaPods search engine repo, please direct CocoaPods issues to the main CocoaPods repository.

25 stars 11 forks source link

Implement stemming of words #79

Closed kylef closed 9 years ago

kylef commented 9 years ago

We should figure the root form of all the words when indexing and searching.

There are a few benefits of this:

Marginally less memory will be needed to store the stemmed version of the word when indexing (every little bit helps)
We will be able to get consistent results regardless of the pluralisation (and other factors) of search terms.
- (header vs headers)
- (heading vs head)
- etc
It gives @floere a fun task to do over the weekend.

floere commented 9 years ago

Nitpick: Not the root form, the stem, actually – hence stemming :)

floere commented 9 years ago

Sadly, as we do not search full text, we cannot easily use stemming. We can't just stem all search terms, as for example a snowball stemmer will think "ios" is "io" stemmed.

What I'd have to do for Picky is implement stemming/non-stemming per searched category.

floere commented 9 years ago

See https://github.com/floere/picky/issues/138 (closed)

floere commented 9 years ago

Done and released. Check the summary of PonyDebugger: http://cocoapods.org/?q=summary:networking

AliSoftware commented 9 years ago

Awesome! :+1: :dancer: