Umlauts are not being displayed in call histories

[huzun]

Entries (Hüseyin Uzun is displayed as Hseyin Uzun) with umlauts are not being displyed in call histories (GUI and Snom-Phone 870)

[spag]

This is not a call history but a caller-ID issue as many phones are unable to display any sort of non ASCII characters as caller-ID.

[wintermeyer]

IMO http://api.rubyonrails.org/classes/ActiveSupport/Inflector.html#method-i-transliterate is the way to go for solving this problem.

[spag]

We cannot do this in the dialplan or on a per call basis then. We need to add a caller_id field in the database for all relevant record types (e.g. PhoneBookEntry, AreaCode) to be filled after the corresponding record is saved.

[duebel-]

I think this is a general issue in the lua database connection. The umlauts are missing in the freeswitch.log and fs_cli, too.

[duebel-]

Please review this patch:

diff --git a/misc/freeswitch/scripts/common/str.lua b/misc/freeswitch/scripts/common/str.lua
index 3fd8fde..32d4bf9 100644
--- a/misc/freeswitch/scripts/common/str.lua
+++ b/misc/freeswitch/scripts/common/str.lua
@@ -69,7 +69,20 @@ end

 -- remove special characters
 function to_ascii(value)
-  return (to_s(value):gsub('[^A-Za-z0-9%-%_ %(%)]', ''));
+  local lookuptable = { ['\246'] = 'ö',
+                        ['\228'] = 'ä',
+                        ['\252'] = 'ü',
+                        ['\196'] = 'Ä',
+                        ['\214'] = 'Ö',
+                        ['\220'] = 'Ü',
+                        ['\223'] = 'ß' };
+  local valuestr = to_s(value);
+  local k, v;
+  for k, v in pairs(lookuptable) do
+    valuestr = valuestr:gsub(k, v)
+  end
+
+  return (valuestr:gsub('[^A-ZÄÖÜa-zäöüß0-9%-%_ %(%),]', ''));
 end

 -- to SQL

It works for German umlauts.

Feel free to integrate it.

[duebel-]

Now the call display is showing umlauts but the call histories have strange multibyte characters.

I'll dig deeper...

[duebel-]

I couldn't get the multi byte characters back to database. I backpedal one step:

diff --git a/misc/freeswitch/scripts/common/str.lua b/misc/freeswitch/scripts/common/str.lua
index 3fd8fde..fa940b9 100644
--- a/misc/freeswitch/scripts/common/str.lua
+++ b/misc/freeswitch/scripts/common/str.lua
@@ -69,7 +69,20 @@ end

 -- remove special characters
 function to_ascii(value)
-  return (to_s(value):gsub('[^A-Za-z0-9%-%_ %(%)]', ''));
+  local lookuptable = { ['\246'] = 'oe',
+                        ['\228'] = 'ae',
+                        ['\252'] = 'ue',
+                        ['\196'] = 'Ae',
+                        ['\214'] = 'Oe',
+                        ['\220'] = 'Ue',
+                        ['\223'] = 'ss' };
+  local valuestr = to_s(value);
+  local k, v;
+  for k, v in pairs(lookuptable) do
+    valuestr = valuestr:gsub(k, v)
+  end
+
+  return (valuestr:gsub('[^A-Za-z0-9%-%_ %(%),]', ''));
 end

 -- to SQL