search

dakrone / clj-http

An idiomatic clojure http client wrapping the apache client. Officially supported version.
http://clojars.org/clj-http
MIT License
1.79k stars 410 forks source link

Skip query param encoding? #605

Open jeaye opened 2 years ago

jeaye commented 2 years ago

I'm using css.gg to bring in CSS for icons, as mentioned here: https://github.com/astrit/css.gg#3-collection

Currently, I'm trying to port a query from HTML to CLJ. As an HTML stylesheet link, this works fine:

<link href='https://css.gg/css?=link|bulb' rel='stylesheet'>

Note that the | is not encoded to %7C. However, if I try this in clj-http, it's always encoded, which breaks the request, since css.gg is expecting the | separator.

I've tried these forms:

(clj-http.client/get "https://css.gg/css?=link|bulb") ; requests /?=link%7Cbulb
(clj-http.client/get "https://css.gg/css" {:query-params {"" "link|bulb"}}) ; requests /?=link%7Cbulb

It may appear to be working, if you try it, but that's only because it's returning the CSS for link alone. Since it's not seeing the |, it doesn't return the CSS for bulb. Finally, note that this works just fine with curl:

$ curl "https://css.gg/css?=link|bulb"

Can I do this with clj-http?

jeaye commented 2 years ago

After some digging through the code and tinkering, I've found that, even if I disable the URL encoding, this will ultimately result in Java-related URI parsing errors.

(with-redefs [clj-http.client/url-encode-illegal-characters identity]
  (:body (clj-http.client/get "https://css.gg/css?=link|bulb")))

; eval (current-form): (with-redefs [clj-http.client/url-encode-illegal-c...
; (err) Execution error (URISyntaxException) at java.net.URI$Parser/fail (URI.java:2913).
; (err) Illegal character in query at index 24: https://css.gg/css?=link|bulb

So it could be that this just can't be done with the java.net classes.

EDIT: (URL. "https://css.gg/css?=link|bulb") works fine. Just not if you try to run it through a parser.

krydos commented 2 years ago

hey @jeaye just out of curiosity, did you try to use different separator like ,

(clj-http.client/get "https://css.gg/css?=link,bulb")

this one seems like returning what's expected.

I know it's undocumented feature but I just somehow happened to find it. /, \, . seems working too btw.

jeaye commented 2 years ago

Oh, that's amazing. Thank you!