GZIP support for getting contents

[GoogleCodeExporter]

Apparently there is no GZIP use when querying the WikiMedia's API, and that
would be very important for projects like Wikipedia and Wikitravel.
It's easy to implement:
Given an instance of URLConnection u:
// enable gzip compression
u.setRequestProperty("Accept-encoding", "gzip");

Best Regards,
Diego.

Original issue reported on code.google.com by diego....@gmail.com on 1 Mar 2010 at 8:13

[GoogleCodeExporter]

Original comment by axelclk@gmail.com on 1 Mar 2010 at 10:26

• Changed state: Accepted

[GoogleCodeExporter]

Well, I've made the changes in my local read-only copy, so here they are:
Sorry for not having done this in a better way (diff file or similar)

File Connector.java:

   /**
     * Format the response body as XML String. Especially for some obscure <a
     * href="http://en.wikipedia.org/wiki/Byte_order_mark ">byte order mark</a>
     * cases. See <a
     * href="http://code.google.com/p/gwtwiki/issues/detail?id=33">Issue #33</a>
     * 
     * @param method
     * @return
     * @throws IOException
     */
    public static String getAsXmlString(HttpMethod method) throws IOException {
        Header contentEncoding = method.getResponseHeader("Content-Encoding");
        InputStream instream = method.getResponseBodyAsStream();
        if (contentEncoding != null
                && contentEncoding.getValue().equalsIgnoreCase("gzip")) {
            instream = new GZIPInputStream(instream);
        }
        String responseBody = convertStreamToString(instream);
        if (responseBody.length() > 0 && responseBody.charAt(0) != '<') {
            // try to find XML.
            int indx = responseBody.indexOf("<?xml");
            if (indx > 0) {
                responseBody = responseBody.substring(indx);
            }
        }
        return responseBody;
    }

    private static String convertStreamToString(InputStream is) throws IOException {
        /*
         * To convert the InputStream to String we use the BufferedReader.readLine()
         * method. We iterate until the BufferedReader return null which means
         * there's no more data to read. Each line will appended to a StringBuilder
         * and returned as String.
         */
        if (is != null) {
            StringBuilder sb = new StringBuilder();
            String line;

            try {
                BufferedReader reader = new BufferedReader(new InputStreamReader(is,
"UTF-8"));
                while ((line = reader.readLine()) != null) {
                    sb.append(line).append("\n");
                }
            } finally {
                is.close();
            }
            return sb.toString();
        }
        return "";
    }

  private String executeHttpMethod(HttpMethod method) {
        try {
            method.addRequestHeader("Accept-encoding", "gzip");
            int responseCode = client.executeMethod(method);
            if (responseCode == HttpStatus.SC_OK) {
                return getAsXmlString(method);
            }
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        } catch (HttpException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            method.releaseConnection();
        }
        return null;
    }

Original comment by diego....@gmail.com on 2 Mar 2010 at 8:37

[GoogleCodeExporter]

Adde your patch. See.
http://code.google.com/p/gwtwiki/source/detail?r=1092

Original comment by axelclk@gmail.com on 2 Mar 2010 at 10:14

[GoogleCodeExporter]

Original comment by axelclk@gmail.com on 13 May 2010 at 3:27

• Changed state: Fixed