Closed j0rd1smit closed 2 years ago
cc: @lxning @msaroufim
@j0rd1smit
How do I format a REST API request if the model expects multiple inputs?
Example:
requests.post("http://localhost:8080/predictions/squeezenet1_1", files={'data': open('docs/images/dogs-before.jpg', 'rb'), 'data': open('docs/images/kitten_small.jpg', 'rb')})
Is it possible to send multiple instances in single rest requests? No.
Is it possible to use a custom handler to accept, for example, a JSON body in the request? Yes. Torchserve envelope support it.
@j0rd1smit Please feel free to reopen this ticket if any further help is needed.
š Documentation: Are custom JSON bodies possible with custom handlers? e.g., multiple instances per request, multi-modality instances, etc.
I have been reading the documentation, but I'm still left with the following questions:
1) How do I format a REST API request if the model expects multiple inputs? For example, the model expects two or more images/tensors as input. Should this be done by uploading multiple files in the request?
2) Is it possible to send multiple instances in single rest requests? In TensorFlow-Serving, it is possible to send multiple instances in the same request as if the JSON body has the format:
Is there something similar possible in PyTorch-Serve?
3) Is it only possible to send input instances via the REST API as files? In all the documentation and examples I have seen, all the instances are sent via file uploads using either
curl URL -T file_path
orrequests.post(URL, files={...})
. I was wondering if it is possible to use a custom handler to accept, for example, a JSON body in the request. This would make it possible to solve questions 1 and 2 using a custom handler.