Incorrect treatement of named html entities

[GoogleCodeExporter]

Hello ,

some days ago we found out that a very strange behavior of our software was 
caused by the implementation of method decodeForHTML  as defined in interface 
org.owasp.esapi.Encoder.
In detail, the concrete implementation (class HTMLEntityCodec) tries to decode 
HTML encoded text by finding out HTML entity parts of string literals first and 
then trying to find a corresponding entry in a map (class HTMLEntityCodec, 
method getNamedEntity).

An example: Input (HML encoded) text: "abcDefG&|Uuml;xyz"
Now the parts "abcDefG" and "xyz" pass entity check und won't be modified, 
whereas the part "&|Uuml;" will be recognized being an HTML entity.
As a result of this, the part "&|Uuml;" will be handed over to the method 
getNamedEntity, which now tries to get a corresponding entry for this named 
HTML entity (e.g. method should return "<" for "&|lt;").

In my opinion, this method does not work correctly due to the fact, that input 
will be converted to lower case which leads to incorrect output if you use case 
sensitive HTML entities like "&|Uuml;" (=Ü), "&|uuml;" (=ü).
This results in an incorrect output "ü" for input "&|Uuml;" but should be "Ü" 
(upper case!)

Also, this method (in class HTMLEntityCodec) uses a hard coded map for lookup 
even though there also exists a property file named antisamy-esapi.xml which 
also defines HTML entities.

Code:

      private Character getNamedEntity( PushbackString input ) {
            // ...
            len = Math.min(input.remainder().length(), entityToCharacterTrie.getMaxKeyLength());
            for(int i=0;i<len;i++)
                  possible.append(Character.toLowerCase(input.next()));           // *** problem! ***
            // look up the longest match
            entry = entityToCharacterTrie.getLongestMatch(possible);

What version of the product are you using? On what operating system?

esapi-2.0.1

Does this issue affect only a specified browser or set of browsers?

Affect on a set of browsers

Thank you

Original issue reported on code.google.com by thomas.m...@gmail.com on 26 Jul 2012 at 7:25