rstudio / plumber

Turn your R code into a web API.
https://www.rplumber.io
Other
1.38k stars 256 forks source link

Does plumber support gzipped files? #429

Open konradsemsch opened 5 years ago

konradsemsch commented 5 years ago

Hello!

there's been a recent discussion on the R Studio Community forum (link) about timing differences in local vs. remote API calls. One of the conclusions was that when the response is large it could be a good idea to compress is with gzip or so.

I wanted to follow-up and ask whether it's also possible for plumber to accept gzipped files in requests? I tried a couple of solutions but none of them worked and couldn't find any information in docs or online. Could you provide some minimal example that illustrates that functionality? A piece of advice on that would be very helpful!

schloerke commented 5 years ago

Check out https://github.com/jcpsantiago/protoplumb-playground/blob/9e677bcb07f46ba4063ea768c0cdbf6239e8c90a/prototest/plumber.R#L23 from #386 for an example on how protobuf is decompressed for all incoming requests.

I'm going to guess there's a header value set that can determine if the input value should be unzipped.

There is a lot of room for improvement on how plumber handles post body inputs.


Also, if you're doing R to R communication, check out serializer_rds (master branch). It's very small and keeps native objects.

konradsemsch commented 5 years ago

Thanks for your response! So in your view, at the current stage of plumber development, what would be the recommended path to compress the response and request in a way that the API could communicate with other languages? Both the inputs and outputs of our APIs are large.

Could you perhaps make a small example of how such CURLs would look like using protobuf and the rds serializer?

jcpsantiago commented 4 years ago

@konradsemsch from the code @schloerke mentioned above at https://github.com/jcpsantiago/protoplumb-playground :

echo "a : 123" | \
protoc --encode=protoplumb.TestPayload prototest.proto | \
curl -v --header "Content-Type: application/x-protobuf; messagetype=protoplumb.TestPayload" -X POST --data-binary @- http://localhost:8000/echo | \
protoc --decode=protoplumb.TestPayload prototest.proto

## expected output:
## a: 123

this is how you would send a binary request to a plumber api running a protobuf filter (see https://github.com/ozean12/protopretzel for an implementation)

I take the messagetype header to get the correct proto file.