I am calling a REST API which I cannot disclose, but it returns a response with Content-Encoding: deflate.
scala> requests.get(url = redactedUrl, params = redactedParams)
java.util.zip.ZipException: incorrect header check
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:164)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at geny.Internal$.transfer0(Internal.scala:40)
at geny.Internal$.transfer(Internal.scala:61)
at geny.Readable.$anonfun$writeBytesTo$2(Writable.scala:64)
at geny.Readable.$anonfun$writeBytesTo$2$adapted(Writable.scala:64)
at requests.Requester$$anon$1.processWrappedStream$1(Requester.scala:338)
at requests.Requester$$anon$1.readBytesThrough(Requester.scala:345)
at geny.Readable.writeBytesTo(Writable.scala:64)
at geny.Readable.writeBytesTo$(Writable.scala:64)
at requests.Requester$$anon$1.writeBytesTo(Requester.scala:165)
at requests.Requester.apply(Requester.scala:114)
... 40 elided
The issue here essentially boils down to this stackoverflow post. The real bug is probably with the REST API that I'm calling, but it should be said that regular browsers, curl, wgetand Python Requests can handle this discrepancy. The culprit is most likely some misnamed PHP function.
A workaround is to manually handle decompression. Something akin to this:
def inflateResponse(r: geny.Readable): String = {
val inflater = new Inflater(true)
val out = new ByteArrayOutputStream()
val inflaterOut = new InflaterOutputStream(out, inflater)
try {
r.writeBytesTo(inflaterOut)
out.toString("UTF-8")
}
finally {
inflaterOut.close()
}
}
val resp = requests.get(url = redactedUrl, params = redactedParams, autoDecompress = false)
inflateResponse(resp)
// or preferably
val str = requests.get.stream(url = redactedUrl, params = redactedParams, autoDecompress = false)
inflateResponse(str)
Or alternatively set headers to Map("Accept-Encoding" -> "gzip") if the server supports it, and avoid the deflate nonsense.
ID: 71
Original Author: Jasper-M
I am calling a REST API which I cannot disclose, but it returns a response with
Content-Encoding: deflate.The issue here essentially boils down to this stackoverflow post. The real bug is probably with the REST API that I'm calling, but it should be said that regular browsers,
curl,wgetand Python Requests can handle this discrepancy. The culprit is most likely some misnamed PHP function.A workaround is to manually handle decompression. Something akin to this:
Or alternatively set
headerstoMap("Accept-Encoding" -> "gzip")if the server supports it, and avoid thedeflatenonsense. ID: 71 Original Author: Jasper-M