decode from encoded string by IE

[asfimport]

Takayuki Kaneko (Bug 38115): java.net.URLDecoder sometimes can't handle decoding correctry. It depends on behaviour of IE's encoding.

But Tomcat can handle decoding correctry. So I made a patch of decoding in reference to Tomcat decoding (org.apache.catalina.util.RequestUtil class).

Regards,

Severity: normal OS: other

Duplicated by:

• https://github.com/apache/jmeter/issues/2118

Blocks:

• https://github.com/apache/jmeter/issues/2118

[asfimport]

Takayuki Kaneko (migrated from Bugzilla): Created attachment patch.txt: patch forJOrphanUtils

patch.txt

````diff Index: C:/eclipse_workspaces/w_jmeter/trunk/src/jorphan/org/apache/jorphan/util/JOrphanUtils.java =================================================================== --- C:/eclipse_workspaces/w_jmeter/trunk/src/jorphan/org/apache/jorphan/util/JOrphanUtils.java (revision 359489) +++ C:/eclipse_workspaces/w_jmeter/trunk/src/jorphan/org/apache/jorphan/util/JOrphanUtils.java (working copy) @@ -234,21 +234,82 @@ * @return the encoded string */ public static String decode(String string, String encoding) throws UnsupportedEncodingException { - if (decodeMethod != null) { - // JDK1.4: return URLDecoder.decode(string, encoding); - Object args[] = { string, encoding }; + if (string == null) + return (null); + + // use the specified encoding to extract bytes out of the + // given string so that the encoding is not lost. If an + // encoding is not specified, let it use platform default + byte[] bytes = null; + try { + if (encoding == null) { + bytes = string.getBytes(); + } else { + bytes = string.getBytes(encoding); + } + } catch (UnsupportedEncodingException uee) { + } + + return URLDecode(bytes, encoding); + + } + + /** + * Decode and return the specified URL-encoded byte array. + * + * @param bytes + * The url-encoded byte array + * @param enc + * The encoding to use; if null, the default encoding is used + * @exception IllegalArgumentException + * if a '%' character is not followed by a valid 2-digit + * hexadecimal number + */ + public static String URLDecode(byte[] bytes, String enc) { + + if (bytes == null) + return (null); + + int len = bytes.length; + int ix = 0; + int ox = 0; + while (ix < len) { + byte b = bytes[ix++]; // Get byte to test + if (b == '+') { + b = (byte) ' '; + } else if (b == '%') { + b = (byte) ((convertHexDigit(bytes[ix++]) << 4) + convertHexDigit(bytes[ix++])); + } + bytes[ox++] = b; + } + if (enc != null) { try { - return (String) decodeMethod.invoke(null, args); + return new String(bytes, 0, ox, enc); } catch (Exception e) { log.warn("Error trying to decode", e); - return string; } - } else { - return URLDecoder.decode(string); // JDK1.3 } + return new String(bytes, 0, ox); + } - /** + /** + * Convert a byte character value to hexidecimal digit value. + * + * @param b + * the character value byte + */ + private static byte convertHexDigit(byte b) { + if ((b >= '0') && (b <= '9')) + return (byte) (b - '0'); + if ((b >= 'a') && (b <= 'f')) + return (byte) (b - 'a' + 10); + if ((b >= 'A') && (b <= 'F')) + return (byte) (b - 'A' + 10); + return 0; + } + + /** * Simple-minded String.replace() for JDK1.3 Should probably be recoded... * * @param source ````

[asfimport]

Sebb (migrated from Bugzilla): Under what circumstances does java.net.URLDecoder not work correctly?

Can you provide some examples that we can add to a test case?

[asfimport]

Takayuki Kaneko (migrated from Bugzilla): This JSP is tested on Tomcat5.0.30 and WindowsXP(Japanese).

Created attachment test.jsp: Encoding Test JSP Example

test.jsp

```` <%@ page language="java" pageEncoding="Windows-31J" contentType="text/html;charset=Windows-31J" %> <% String str = null; String query = null; if (request.getMethod().equals("GET")) { query = request.getQueryString(); if (query != null) { str = java.net.URLDecoder.decode(query.substring(query.indexOf("=") + 1), "Windows-31j"); } else { str = "�o�^"; } } else if (request.getMethod().equals("POST")) { request.setCharacterEncoding("Windows-31J"); str = request.getParameter("str"); } %>

This JSP tests decoding methods.

If you push post button, string in textbox will be decoded by org.apache.catalina.util.RequestUtil.

If you push get button, string in textbox will be decoded by java.util.URLDecoder.

Some characters aren't decoded correctly. It is caused by mismatch between IE encoding and URLDecoder decoding.

---

You can copy&paste following Japanese Characters and push button!

�o�^

(URLEncoder.encode result=<%=java.net.URLEncoder.encode("�o�^", "Windows-31J")%>)

���O

(URLEncoder.encode result=<%=java.net.URLEncoder.encode("���O", "Windows-31J")%>)

---

String

getQueryString: <%=query%>

Decoded String: <%=str%>

````

[asfimport]

Takayuki Kaneko (migrated from Bugzilla): Hi Sebb,

Thank you for replying.

I attached new JSP Example for encode/decode tests. I expect you will be able to understand circumstances on java.net.URLDecoder. Please check it out.

Regards,

[asfimport]

Tetsuya Takatsuru (migrated from Bugzilla): org.apache.commons.codec.net.URLCodec#encode/decode is better than java.net.URLEncoder/URLDecoder.

java.net.URLDecoder cannot decode correctly in this case:

===example String s = "登録";

String commons_encoded = new URLCodec("MS932").encode(s); System.out.println("Encoded by commons-codec: " + commons_encoded); // outputs: Encoded by commons-codec: %93o%98%5E // some CJK multi-byte encodings use ASCII area at second byte. // IE and Firefox (and some other browsers) encodes like as this.

String java_encoded = URLEncoder.encode(s, "MS932"); System.out.println("Encoded by java.net.URLEncoder: " + java_encoded); // outputs: Encoded by java.net.URLEncoder: %93%6F%98%5E

System.out.println(URLDecoder.decode(java_encoded, "MS932")); //OK System.out.println(URLDecoder.decode(commons_encoded, "MS932")); //NG System.out.println(new URLCodec("MS932").decode(java_encoded)); //OK System.out.println(new URLCodec("MS932").decode(commons_encoded)); //OK ===end example

[asfimport]

Tetsuya Takatsuru (migrated from Bugzilla): Created attachment JMeter_URLCodec.patch: using Commons URLCodec instead of java.net.URLDecoder/Encoder

[asfimport]

Tetsuya Takatsuru (migrated from Bugzilla): Created attachment Screenshot.png: screenshot when running my example

[asfimport]

@pmouawad (migrated from Bugzilla): This bug has been marked as a duplicate of https://github.com/apache/jmeter/issues/2118