search

rybalkinsd / kohttp

Kotlin DSL http client
https://kohttp.gitbook.io
Apache License 2.0
478 stars 42 forks source link

possible performance problems #200

Open alosdev opened 4 years ago

alosdev commented 4 years ago

https://github.com/rybalkinsd/kohttp/blob/188b5c5a6e770a167fabcddc928dd4978af2a230/kohttp-jackson/src/main/kotlin/io/github/rybalkinsd/kohttp/jackson/ext/ResponseExt.kt#L69

in the following line, the library makes a string out of the response and it loses the purpose of a stream reader. Would it be possible to use the InpputStream to leverage the way how Jackson is working? Also with very large responses, it would raise some memory pressure.

rybalkinsd commented 4 years ago

Thanks for posting this issue @alosdev!

Overall, It's a great idea to introduce a simplified API to work with response streaming.

Indeed, in the quoted file we load the whole response. The main idea of .toType extension is to deserialise simple/small bodies as type T

val simple: SimpleClass? = it.toType()

In this case we have an extra allocation(for tmp string) + sequential processing vs streaming processing overhead. In my mind it would be great to have a benchmark to make a decision. Do you have any use case in your mind?

From my understanding getting deserialized type T or JsonNode itself will not radically change the situation (the whole object will be in memory after parsing) However parsing arrays of to Stream<T> or fetching a particular filed(sub-object) from a huge body could help. Do you have any thoughts about API?