olivernn / lunr.js

A bit like Solr, but much smaller and not as bright
http://lunrjs.com
MIT License
8.96k stars 548 forks source link

stack denying tokens #123

Closed mobinseven closed 9 years ago

mobinseven commented 9 years ago

I figured out that in 'lunr.Pipeline.prototype.run' this line :

token = this._stack[j](token, i, tokens)

is returning null or undefined for non-english charachters (Persian [farsi] in my case). Any Ideas?

olivernn commented 9 years ago

see #122 for more details