search

dakrone / clj-http

An idiomatic clojure http client wrapping the apache client. Officially supported version.
http://clojars.org/clj-http
MIT License
1.78k stars 408 forks source link

json coercion failed with cheshire 5.8.1 #555

Closed jngbng closed 4 years ago

jngbng commented 4 years ago

It is not really a bug, but I though someone would make a mistake as I did.

As of clj-http 3.10.1, it depends on cheshire 5.9.0. (https://github.com/dakrone/clj-http/blob/3.x/changelog.org#3101).

JSON parsing is always strict. See README#Incrementally JSON Parsing. Requires cheshire >= 5.9.0.

I thought it would not affect me since I do not use those new feature. But actually, clj-http is NOT compatible with cheshire 5.8.1 AT ALL: it throws NullPointerException on runtime. (I missed this problem when I test on my local environment due to lein cache bug.)

It would be better that this version requirement is mentioned as a breaking change.

I hope this helps someone. :)

Following is a sample code.

project.clj:

(defproject clj-http-exception "0.1.0-SNAPSHOT"
  :dependencies [[org.clojure/clojure "1.10.1"]
                 [cheshire "5.8.1"]
                 [clj-http "3.10.2"]]
  :main main
  :source-paths ["src"])

main.clj:

(ns main
  (:gen-class)
  (:require [clj-http.client :as client]))

(defn -main
  [& _m]
  (prn (client/get "https://appleid.apple.com/auth/keys"
                         {:socket-timeout 5000
                          :conn-timeout 5000
                          :as :json})))

result:

{:clojure.main/message
 "Syntax error (NullPointerException) compiling at (/private/var/folders/zn/77sfq94n4259v4jds_6k23yh0000gn/T/form-init2163147138559460184.clj:1:125).\nnull\n",
 :clojure.main/triage
 {:clojure.error/phase :compile-syntax-check,
  :clojure.error/line 1,
  :clojure.error/column 125,
  :clojure.error/source "form-init2163147138559460184.clj",
  :clojure.error/path
  "/private/var/folders/zn/77sfq94n4259v4jds_6k23yh0000gn/T/form-init2163147138559460184.clj",
  :clojure.error/class java.lang.NullPointerException},
 :clojure.main/trace
 {:via
  [{:type clojure.lang.Compiler$CompilerException,
    :message
    "Syntax error compiling at (/private/var/folders/zn/77sfq94n4259v4jds_6k23yh0000gn/T/form-init2163147138559460184.clj:1:125).",
    :data
    {:clojure.error/phase :compile-syntax-check,
     :clojure.error/line 1,
     :clojure.error/column 125,
     :clojure.error/source
     "/private/var/folders/zn/77sfq94n4259v4jds_6k23yh0000gn/T/form-init2163147138559460184.clj"},
    :at [clojure.lang.Compiler load "Compiler.java" 7648]}
   {:type java.lang.NullPointerException,
    :at [clojure.core$apply invokeStatic "core.clj" 665]}],
  :trace
  [[clojure.core$apply invokeStatic "core.clj" 665]
   [clojure.core$apply invoke "core.clj" 660]
   [clj_http.client$json_decode invokeStatic "client.clj" 131]
   [clj_http.client$json_decode doInvoke "client.clj" 127]
   [clojure.lang.RestFn invoke "RestFn.java" 421]
   [clj_http.client$decode_json_body invokeStatic "client.clj" 457]
   [clj_http.client$decode_json_body invoke "client.clj" 449]
   [clj_http.client$coerce_json_body invokeStatic "client.clj" 465]
   [clj_http.client$coerce_json_body doInvoke "client.clj" 460]
   [clojure.lang.RestFn invoke "RestFn.java" 445]
   [clj_http.client$eval4369$fn__4370 invoke "client.clj" 527]
   [clojure.lang.MultiFn invoke "MultiFn.java" 234]
   [clj_http.client$output_coercion_response
    invokeStatic
    "client.clj"
    561]
   [clj_http.client$output_coercion_response invoke "client.clj" 558]
   [clj_http.client$wrap_output_coercion$fn__4413
    invoke
    "client.clj"
    572]
   [clj_http.client$wrap_exceptions$fn__4210 invoke "client.clj" 249]
   [clj_http.client$wrap_accept$fn__4466 invoke "client.clj" 726]
   [clj_http.client$wrap_accept_encoding$fn__4473
    invoke
    "client.clj"
    748]
   [clj_http.client$wrap_content_type$fn__4460 invoke "client.clj" 709]
   [clj_http.client$wrap_form_params$fn__4569 invoke "client.clj" 950]
   [clj_http.client$wrap_nested_params$fn__4590
    invoke
    "client.clj"
    984]
   [clj_http.client$wrap_flatten_nested_params$fn__4599
    invoke
    "client.clj"
    1008]
   [clj_http.client$wrap_method$fn__4527 invoke "client.clj" 884]
   [clj_http.cookies$wrap_cookies$fn__426 invoke "cookies.clj" 131]
   [clj_http.links$wrap_links$fn__1943 invoke "links.clj" 63]
   [clj_http.client$wrap_unknown_host$fn__4607
    invoke
    "client.clj"
    1037]
   [clj_http.client$request_STAR_ invokeStatic "client.clj" 1165]
   [clj_http.client$request_STAR_ invoke "client.clj" 1158]
   [clj_http.client$get invokeStatic "client.clj" 1171]
   [clj_http.client$get doInvoke "client.clj" 1167]
   [clojure.lang.RestFn invoke "RestFn.java" 423]
   [main$_main invokeStatic "main.clj" 7]
   [main$_main doInvoke "main.clj" 5]
   [clojure.lang.RestFn invoke "RestFn.java" 397]
   [clojure.lang.Var invoke "Var.java" 380]
   [user$eval140 invokeStatic "form-init2163147138559460184.clj" 1]
   [user$eval140 invoke "form-init2163147138559460184.clj" 1]
   [clojure.lang.Compiler eval "Compiler.java" 7177]
   [clojure.lang.Compiler eval "Compiler.java" 7167]
   [clojure.lang.Compiler load "Compiler.java" 7636]
   [clojure.lang.Compiler loadFile "Compiler.java" 7574]
   [clojure.main$load_script invokeStatic "main.clj" 475]
   [clojure.main$init_opt invokeStatic "main.clj" 477]
   [clojure.main$init_opt invoke "main.clj" 477]
   [clojure.main$initialize invokeStatic "main.clj" 508]
   [clojure.main$null_opt invokeStatic "main.clj" 542]
   [clojure.main$null_opt invoke "main.clj" 539]
   [clojure.main$main invokeStatic "main.clj" 664]
   [clojure.main$main doInvoke "main.clj" 616]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.lang.Var applyTo "Var.java" 705]
   [clojure.main main "main.java" 40]],
  :phase :compile-syntax-check}}

ps. After upgrading clj-http on project.clj, lein magically uses cheshire 5.9.1 and everything seems to work well. lein deps :tree also shows cheshire 5.9.1 is used. But after lein clean, it stops working. should I always lein clean after updateing project.clj?

rymndhng commented 4 years ago

Hey @jngbng, thanks for the feedback!

That was my bad for not clarifying that the cheshire dependency is a breaking change 🤦. I completely agree that the README could clarify this on the breaking change. I also want to investigate a bit further on whether there's other ways of providing helpful error messages when using optional dependencies.

RE: lein clean. I strongly recommend running lein clean after changing any dependencies. What can happen is that you may have previously compiled clojure namespaces that get picked up. In general, I recommend running lein clean after changing changing dependencies or after running lein jar or lein uberjar. This should cover most of the cases.

rymndhng commented 4 years ago

@jngbng See #558 and let me know if this addresses your concerns.

I've re-ran your test code with a snapshot build and this is now the error message that is returned:

❯ lein run
Syntax error (IllegalStateException) compiling at (/private/var/folders/gr/n1_1kt5126dbgmjjzwm44nf00000gn/T/form-init8169005464871833131.clj:1:125).
Missing #'cheshire.core/parse-stream-strict. Ensure the version of cheshire is >= 3.9.0

Full report at:
/var/folders/gr/n1_1kt5126dbgmjjzwm44nf00000gn/T/clojure-17780328165989224335.edn

Contents of Full Report:

/var/folders/gr/n1_1kt5126dbgmjjzwm44nf00000gn/T/clojure-17780328165989224335.edn ``` clojure {:clojure.main/message "Syntax error (IllegalStateException) compiling at (/private/var/folders/gr/n1_1kt5126dbgmjjzwm44nf00000gn/T/form-init8169005464871833131.clj:1:125).\nMissing #'cheshire.core/parse-stream-strict. Ensure the version of cheshire is >= 3.9.0\n", :clojure.main/triage {:clojure.error/phase :compile-syntax-check, :clojure.error/line 1, :clojure.error/column 125, :clojure.error/source "form-init8169005464871833131.clj", :clojure.error/path "/private/var/folders/gr/n1_1kt5126dbgmjjzwm44nf00000gn/T/form-init8169005464871833131.clj", :clojure.error/class java.lang.IllegalStateException, :clojure.error/cause "Missing #'cheshire.core/parse-stream-strict. Ensure the version of cheshire is >= 3.9.0"}, :clojure.main/trace {:via [{:type clojure.lang.Compiler$CompilerException, :message "Syntax error compiling at (/private/var/folders/gr/n1_1kt5126dbgmjjzwm44nf00000gn/T/form-init8169005464871833131.clj:1:125).", :data {:clojure.error/phase :compile-syntax-check, :clojure.error/line 1, :clojure.error/column 125, :clojure.error/source "/private/var/folders/gr/n1_1kt5126dbgmjjzwm44nf00000gn/T/form-init8169005464871833131.clj"}, :at [clojure.lang.Compiler load "Compiler.java" 7648]} {:type java.lang.IllegalStateException, :message "Missing #'cheshire.core/parse-stream-strict. Ensure the version of cheshire is >= 3.9.0", :at [clj_http.client$json_decode invokeStatic "client.clj" 132]}], :trace [[clj_http.client$json_decode invokeStatic "client.clj" 132] [clj_http.client$json_decode doInvoke "client.clj" 127] [clojure.lang.RestFn invoke "RestFn.java" 421] [clj_http.client$decode_json_body invokeStatic "client.clj" 460] [clj_http.client$decode_json_body invoke "client.clj" 452] [clj_http.client$coerce_json_body invokeStatic "client.clj" 468] [clj_http.client$coerce_json_body doInvoke "client.clj" 463] [clojure.lang.RestFn invoke "RestFn.java" 445] [clj_http.client$eval3186$fn__3187 invoke "client.clj" 530] [clojure.lang.MultiFn invoke "MultiFn.java" 234] [clj_http.client$output_coercion_response invokeStatic "client.clj" 564] [clj_http.client$output_coercion_response invoke "client.clj" 561] [clj_http.client$wrap_output_coercion$fn__3230 invoke "client.clj" 575] [clj_http.client$wrap_exceptions$fn__3027 invoke "client.clj" 252] [clj_http.client$wrap_accept$fn__3283 invoke "client.clj" 729] [clj_http.client$wrap_accept_encoding$fn__3290 invoke "client.clj" 751] [clj_http.client$wrap_content_type$fn__3277 invoke "client.clj" 712] [clj_http.client$wrap_form_params$fn__3386 invoke "client.clj" 953] [clj_http.client$wrap_nested_params$fn__3407 invoke "client.clj" 987] [clj_http.client$wrap_flatten_nested_params$fn__3416 invoke "client.clj" 1011] [clj_http.client$wrap_method$fn__3344 invoke "client.clj" 887] [clj_http.cookies$wrap_cookies$fn__426 invoke "cookies.clj" 131] [clj_http.links$wrap_links$fn__1943 invoke "links.clj" 63] [clj_http.client$wrap_unknown_host$fn__3424 invoke "client.clj" 1040] [clj_http.client$request_STAR_ invokeStatic "client.clj" 1168] [clj_http.client$request_STAR_ invoke "client.clj" 1161] [clj_http.client$get invokeStatic "client.clj" 1174] [clj_http.client$get doInvoke "client.clj" 1170] [clojure.lang.RestFn invoke "RestFn.java" 423] [main$_main invokeStatic "main.clj" 7] [main$_main doInvoke "main.clj" 5] [clojure.lang.RestFn invoke "RestFn.java" 397] [clojure.lang.Var invoke "Var.java" 380] [user$eval140 invokeStatic "form-init8169005464871833131.clj" 1] [user$eval140 invoke "form-init8169005464871833131.clj" 1] [clojure.lang.Compiler eval "Compiler.java" 7177] [clojure.lang.Compiler eval "Compiler.java" 7167] [clojure.lang.Compiler load "Compiler.java" 7636] [clojure.lang.Compiler loadFile "Compiler.java" 7574] [clojure.main$load_script invokeStatic "main.clj" 475] [clojure.main$init_opt invokeStatic "main.clj" 477] [clojure.main$init_opt invoke "main.clj" 477] [clojure.main$initialize invokeStatic "main.clj" 508] [clojure.main$null_opt invokeStatic "main.clj" 542] [clojure.main$null_opt invoke "main.clj" 539] [clojure.main$main invokeStatic "main.clj" 664] [clojure.main$main doInvoke "main.clj" 616] [clojure.lang.RestFn applyTo "RestFn.java" 137] [clojure.lang.Var applyTo "Var.java" 705] [clojure.main main "main.java" 40]], :cause "Missing #'cheshire.core/parse-stream-strict. Ensure the version of cheshire is >= 3.9.0", :phase :compile-syntax-check}} ```
rymndhng commented 3 years ago

The improvement is in 3.10.3