Simplify and test url indexing

commonsearch / cosr-back

Backend of Common Search. Analyses webpages and sends them to the index.

https://about.commonsearch.org

Apache License 2.0

122 stars 24 forks source link

Closed Sentimentron closed 8 years ago

Sentimentron commented 8 years ago

Don't attempt to strip public domain suffixes. Potential fix for #31

Sentimentron commented 8 years ago

Looks like I forgot to delete stuff, in all the various places. Thanks Travis!

sylvinus commented 8 years ago

Hi @Sentimentron! Sorry for the lag on this.

I think we want to keep the get_domain_paid_words() API for now, so I only integrated your tests in master, which confirm the issue was fixed.

Thanks!