request breaks json decoding

[yayitswei]

This request breaks JSON params decoding for me, using [ring-middleware-format "0.3.1"]:

POST /api/session HTTP/1.1
Host: localhost:8080
Content-Length: 36
Accept: application/json, text/plain, */*
Origin: http://localhost:8080
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1631.0 Safari/537.36
Content-Type: application/json;charset=UTF-8
Referer: http://localhost:8080/
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8,es;q=0.6,vi;q=0.4
Cookie: ring-session=0023626c-5f08-4d70-b1e6-c64d5e0c93ca; accountLinkId=BBuipRRmVv-RaWahiPyBXQ

{"link-id":"BBuipRRmVv-RaWahiPyBXQ"}

Throwing this error:

java.io.UnsupportedEncodingException: IBM424_rtl
     at java.lang.StringCoding.decode(StringCoding.java:170)
     at java.lang.String.<init>(String.java:443)
     at java.lang.String.<init>(String.java:515)
     at ring.middleware.format_params$wrap_format_params$fn__663.invoke(format_params.clj:124)
     at ring.middleware.format_response$wrap_format_response$fn__754.invoke(format_response.clj:113)
     at noir.session$noir_session$fn__3757.invoke(session.clj:96)
     at zz.server$wrap_noir_debug_session$fn__1105.invoke(server.clj:73)
     at zz.server$wrap_exception$fn__1114.invoke(server.clj:88)
     at zz.server$wrap_request_logging$fn__1119.invoke(server.clj:102)
     at clojure.lang.Var.invoke(Var.java:415)
     at ring.middleware.keyword_params$wrap_keyword_params$fn__7499.invoke(keyword_params.clj:27)
     at ring.middleware.nested_params$wrap_nested_params$fn__7538.invoke(nested_params.clj:65)
     at ring.middleware.params$wrap_params$fn__7471.invoke(params.clj:55)
     at ring.middleware.multipart_params$wrap_multipart_params$fn__7566.invoke(multipart_params.clj:103)
     at ring.middleware.flash$wrap_flash$fn__3729.invoke(flash.clj:14)
     at ring.middleware.session$wrap_session$fn__3718.invoke(session.clj:43)
     at ring.middleware.cookies$wrap_cookies$fn__3650.invoke(cookies.clj:160)
     at ring.middleware.reload$wrap_reload$fn__7448.invoke(reload.clj:18)
     at org.httpkit.server.HttpHandler.run(RingHandler.java:91)
     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
     at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
     at java.util.concurrent.FutureTask.run(FutureTask.java:138)
     at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
     at java.lang.Thread.run(Thread.java:680)

My workaround was to change ring.middleware.format-params/wrap-format-params to use slurp directly, but I'm not sure if that works in the general case.

(defn wrap-format-params [handler & {:keys [predicate decoder charset handle-error]}]
  (fn [{:keys [#^InputStream body] :as req}]
    (try
      (if (and body (predicate req))
        (let [bstr (slurp body)]
          (if (> (count bstr))
            (let [body (:body req)
                  fmt-params (decoder bstr)
                  req* (assoc req
                              :body-params fmt-params
                              :params (merge (:params req)
                                             (when (map? fmt-params) fmt-params))
                              :body body)]
              (handler req*))
            (handler req)))
        (handler req))
      (catch Exception e
        (handle-error e handler req)))))

[cyruszhang]

same thing happens here. wonder whether this is fixed in 0.3.2?

[cyruszhang]

the request that broke mine:

POST HTTP/1.1 Host: :8080 Content-Type: application/json Cache-Control: no-cache

{"asin": "B003S1UNZA", "marketplace": 1}

[cyruszhang]

turning off guess-charset in format-params/get-or-guess-charset also works; basically ICU4J seems not working properly in guessing some of the encodings

[ngrunwald]

Yes, ICU4J seems sometimes to be rather a random (seems to me it's mostly happening on very short body). I have added in 4c98a32cc3ac5d028a913d9cc69c93cb15592b1d a sanity check to guess-encoding, which should not be able to return unknown encoding anymore. Please let me know if this works for you, and I'll cut a new release quickly.

[cyruszhang]

Hi Nils, it worked for my case. Seems like a safe release to me. Thanks!

[yayitswei]

Thanks for the fix. Could you upload it to Clojars when you get a chance?

[ngrunwald]

Done!