improve quality filter: same text on both sides

hollylspace / hunglish-webapp

Automatically exported from code.google.com/p/hunglish-webapp

1 stars 0 forks source link

improve quality filter: same text on both sides #69

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago

e.g.: search for risque 
http://hunglish.hu/search?huSentence=&enSentence=risque&doc.genre=-10

TODO:
If the sentence is more or less same on both sides then filter them out.
Hint: Use the hash function used in duplicate filter to implement this!

Original issue reported on code.google.com by bpgergo on 6 Jun 2011 at 1:22