Search in FieldTypeCache fields doesn't work with umlauts

[tobaco]

search queries/selectors with FieldCache and umlauts don't work.

e.g. headline = "Steak mit Käse"; fieldcachefield includes headline field

If I search for "Steak" i get the right result. If I search for "Käse" i get nothing.

here's my code:

if($q = $sanitizer->selectorValue($input->get->q)) {
    $input->whitelist('q', $q);
    $matches = $pages->find("fieldcachefield%=$q");
}

In the db table for this fieldcache-field all the umlauts are encoded like this K\u00e4se If I use headline as selector, the right result pops up.

I couldn't find the place where contents of this field get json-encoded, but i would suggest to use the JSON_UNESCAPED_UNICODE flag to leave the umlauts untouched. Although, I don't know if this leads to other drawbacks.

[Da-Fecto]

Sorry, the JSON_UNESCAPED_UNICODE flag is not supported for PHP versions lower then 5.4.0

[teppokoivula]

There are ways to do this for older PHP versions too, though not quite as pretty or effective. I remember having this problem with FieldtypeCache before and ending up with my own, similar, text-only alternative.

This would be cool addition, but unfortunately I'm one of the users running pre-5.4 PHP, so it would only be a breaking change.. unless it's accompanied with an option/check to turn it off and/or pre-5.4 support using other methods :)

[somatonic]

This was also the case when Ryan added the language descriptions for file field. Special chars got encoded, and he fixed it. So I don't know what he did but it's surely possible.

[ryancramerdesign]

PHP 5.3 is now EOL'd. FieldtypeCache is a rarely used Fieldtype (this is the first time this issue has come up for this Fieldtype in 4 years). I'm okay with solving this with PHP 5.4+ JSON flags for those that need it.

On Sat, Sep 6, 2014 at 1:28 PM, Philipp Urlich notifications@github.com wrote:

This was also the case when Ryan added the language descriptions for file field. Special chars got encoded, and he fixed it. So I don't know what he did but it's surely possible.

— Reply to this email directly or view it on GitHub https://github.com/ryancramerdesign/ProcessWire/issues/641#issuecomment-54721558 .

[tobaco]

I'm all in for this change, but i understand the concerns of the other, about breaking the cms. If this could be an option (e.g. in config.php), it would be the best of both worlds.

thanks!

[ryancramerdesign]

This will be added to the next commit in FieldtypeCache.module:

if(defined("JSON_UNESCAPED_UNICODE")) {
  return json_encode($value, JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES); 
} else {
  return json_encode($value); 
}

[tobaco]

THANKS!!!