":star => true" does not include characters in the charset_table

[fabrizioq]

In sphinx.yml

development:
  enable_star: true
  min_prefix_len: 1
  min_infix_len: nil
  character_type: utf-8
  character_table: "0..9, a..z, _, A..Z->a..z, U+00C0->a, U+00C1->a, ..." 
  # table taken from http://yob.id.au/2008/05/08/thinking-sphinx-and-unicode.html

Some search, with accents, the term passed as params[:q]:

Place.search portón, :star => true
Sphinx Query: *port*ó*n*

Place.search óéní, :star => true
Spinx Query: óé*n*í

Am I doing something wrong? Or is this the intended behavior and I should just include the stars myself before passing it to search function?

Thanks a lot.

[pat]

The charset_table is for Sphinx when indexing - but :star => true is for Thinking Sphinx when searching – so there's no direct relation between the two.

The stars are added using a regular expression - and it looks like that expression isn't picking up UTF characters. From a quick test for me, it works in Ruby 1.8.7 but not 1.9.2. Can you confirm that's the case for you as well? I'll try to get it fixed in 1.9.2.

[pat]

Okay, latest commit fixes this - if you're using a Gemfile/Bundler, just grab it from the repo until the next gem is released. Otherwise, you can use this regex instead of true:

Place.search 'portón', :star => /\p{Word}+/u

[fabrizioq]

Thanks a great great lot!

El 10/07/2011, a las 03:05, freelancing-god escribió:

Okay, latest commit fixes this - if you're using Gemfile, just grab it from the repo until the next gem is released. Otherwise, you can use this regex instead of true:

Place.search 'portón', :star => /\p{Word}+/u

Reply to this email directly or view it on GitHub: https://github.com/freelancing-god/thinking-sphinx/issues/251#issuecomment-1540888