Options containing accented letters disappear if html_options is used

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Create the follwing hash in php and display it in a template file.

------------------
$smarty -> assign( "myOptions", array(
                        1800 => 'Joe Schmoe',
                        9904 => 'Andr' . chr( 160 ) . 's',
                        2003 => 'Charlie Brown')
                        );
$smarty -> display( "test.tpl" );
------------------

The chr(161) is a normal ascii character ("á") which is quite common in some 
languages, for example in Hungarian. I used chr() function instead of typing 
'á' in order to avoid character encoding problems if you try to reproduce this 
bug.

2.
Use the following code in test.tpl template:

------------------
{ html_options name=foo options=$myOptions }
------------------

3.
Open the html result in a browser and check the OPTIONs of the produced SELECT. 

What is the expected output? What do you see instead?
I expected to have the three options:
- "Joe Schmoe"
- "András"
- "Charlie Brown"
However, I got the following ones:
- "Joe Schmoe"
- ""
- "Charlie Brown"
As you can see, the option text containing an accented letter disappeared.

The same holds not only for "á" but for several other characters wich are part 
of the ascii character set but they are not a regular utf8 character in 
themselves. 

What version of the product are you using? On what operating system?
Smrty 2.6.27 on WinXP with php 5.4.11

Please provide any additional information below.
The problem is caused by the htmlspecialchars() function called from 
shared.escape_special_chars.php . The point is that from php 5.4 
htmlspecialchars() works in a  different way. It's default encoding has been 
changed. Now it is utf8, and, due to 'András' being not a utf8 text, an empty 
string will be returned. Hence, there is no way to use html_options in non-utf8 
projects. 

Suggested solution: I think it would be nice to have the smarty constructor a 
parameter, or it should be "reconfigured" after creation which encoding to use 
and "ISO-8859-1" should be the default as prior to php 5.4 it was (see: 
http://php.net/manual/en/function.htmlspecialchars.php).

Example:
$s = new Smarty();
$s -> setEncoding( "UTF8" );
$s -> assign( ... );
$s -> display( ... );

or:

$s = new Smarty( "UTF8" );
$s -> assign( ... );
$s -> display( ... );

Original issue reported on code.google.com by csongor....@gmail.com on 18 Feb 2013 at 2:53

[GoogleCodeExporter]

> The chr(161) is a normal ascii character ("á") which is quite common in some 
languages

No it's not ASCII, it's ISO-8859-1. ASCII doesn't contain any chars above 127.

Smarty can change charset using the $_CHARSET static class property, which I 
think is what you're looking for:

Smarty::$_CHARSET = 'utf-8';

That property is used everywhere, including in htmlspecialchars.

Smarty docs on charsets are here: 
http://www.smarty.net/docs/en/charset.tpl#charset.encoding but it's out of date 
and doesn't match what's in the code, though it will still work.

You need to have a pretty good reason not to use UTF-8 these days.

Original comment by marcus.b...@gmail.com on 24 Sep 2013 at 12:31

[GoogleCodeExporter]

Original comment by Uwe.Tews@googlemail.com on 7 Nov 2013 at 9:53

• Changed state: Invalid