Issues while validating locale specific characters.

[GoogleCodeExporter]

What steps will reproduce the problem?
1. I have locale specific site with russia,hungary, etc.. 
2. there will be special character with locale specific.
3. for english locale its working fine, but locale specific instance it creates 
problem while validating. it is not accepting special characters.

What is the expected output? What do you see instead?
It should accept locale specific characters like russian and hungary....

What version of the product are you using? On what operating system?
ESAPI-2.0.jar + windows XP

Please provide any additional information below.
Please let me know what we can do for the same.

Original issue reported on code.google.com by tejas.ma...@gmail.com on 30 Sep 2010 at 4:41

[GoogleCodeExporter]

My point here is that, during internationalization how it will work.

Original comment by tejas.ma...@gmail.com on 30 Sep 2010 at 6:10

[GoogleCodeExporter]

I am getting below error while entering input value for russia locale. 
Input value = Покупайте подгузники, собирайте 
баллы и получайте гарантированные подарки 

13:43:34,670  INFO ExampleApplication:ESAPIFilter:294 - [Anonymous:null@unknown 
-> 127.0.0.1:8080/ExampleApplication/ESAPIFilter] GET 
http://localhost:8080/test/test.jsp
13:43:40,389  INFO ExampleApplication:ESAPIFilter:294 - 
[Anonymous:312555@unknown -> 127.0.0.1:8080/ExampleApplication/ESAPIFilter] 
POST http://localhost:8080/test/test1.jsp?lastname=&Submit=Submit 
Query&firstname=Покупайте подгузники, собирайте 
баллы и получайте гарантированные подарки
13:43:40,420  WARN ExampleApplication:IntrusionDetector:294 - 
[Anonymous:312555@unknown -> 
127.0.0.1:8080/ExampleApplication/IntrusionDetector] Invalid input: 
context=HTTP request parameter: firstname, 
type(HTTPParameterValue)=^[a-zA-Z0-9.!:;@#$%^&{}\[\]()_+\-=,.~'` ]*$, 
input=Покупайте подгузники, собирайте баллы и 
получайте гарантированные подарки
org.owasp.esapi.errors.ValidationException: HTTP request parameter: firstname: 
Invalid input. Please conform to regex ^[a-zA-Z0-9.!:;@#$%^&{}\[\]()_+\-=,.~'` 
]*$ with a maximum length of 65535
    at org.owasp.esapi.reference.validation.StringValidationRule.getValid(StringValidationRule.java:121)
    at org.owasp.esapi.reference.DefaultValidator.getValidInput(DefaultValidator.java:172)
    at org.owasp.esapi.reference.DefaultValidator.assertIsValidHTTPRequest(DefaultValidator.java:692)
    at org.owasp.esapi.reference.DefaultValidator.isValidHTTPRequest(DefaultValidator.java:662)
    at com.org.esapi.ESAPIFilterJava.doFilter(ESAPIFilterJava.java:84)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
    at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
    at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
    at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
    at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)

Original comment by tejas.ma...@gmail.com on 30 Sep 2010 at 8:15

[GoogleCodeExporter]

My current value for Validator.HTTPParameterValue is as below

Validator.HTTPParameterValue=^[a-zA-Z0-9.\\-\\/:;,.?#&+=@_ ]*$

Original comment by tejas.ma...@gmail.com on 30 Sep 2010 at 8:33

[GoogleCodeExporter]

The Validators are not configured out-of-the-box to support 
internationalization. In order to support this you will need to customise the 
regexp patterns to allow for "valid" characters in languages that you are 
supporting. 

In the future it may be worthwhile to create a I18N compatible validation 
framework as a plugin for validators, however - this would require analysts 
with knowledge of what chars are considered valid across any language we 
support with the plugin, or conversely the plugin would use configuration to 
allow end-users to configure valid character sets. 

IMHO - this is beyond the scope of the 2.0 release and the 2.1 release as well, 
so I am slotting this for 2.2 for the time being

Original comment by chrisisbeef on 1 Nov 2010 at 4:07

• Changed state: Accepted
• Added labels: Milestone-Release2.2, Type-Enhancement, Usability
• Removed labels: Type-Defect

[GoogleCodeExporter]

Hi chrisisbeef, 

Thanks for your reply.
However we would like to go with below locales for validating characters.

1) Japan
2) Russia
3) China
4) Arabic
5) Hungary
6) French
7) Dutch
8) Finnish
9) Swedish
and other middle east country specific languages also.

Original comment by tejas.ma...@gmail.com on 2 Nov 2010 at 5:42