I found an interesting issue when trying to use this example with Endpoint Data Capture or with Async Endpoints:
The example notebooks use the JSONDeserializer to fetch JSON responses, but leave the predictor serializer as the default (which for PyTorch is NumpySerializer). As a result, the payload passed to the endpoint is not actually JPEG file data, but a JPEG byte array packaged in a NumPy array.
This functionally works, but it means that the data on the wire is kind of an awkward format: If you deploy the endpoint as Async, the object that gets saved to S3 is not actually a JPEG/PNG/etc image, but something that needs a bit of magic to open. Likewise if you set up a real-time endpoint with data capture, the format of the captured data is not ideal.
Suggested updates
I suggest updating the notebooks to use a raw data serializer such as the DataSerializer (which can accept either data or filenames), which will also require slightly tweaking the inference.py input_fn code to parse the image data as expected.
Hi folks, nice sample!
Problem description
I found an interesting issue when trying to use this example with Endpoint Data Capture or with Async Endpoints:
The example notebooks use the
JSONDeserializerto fetch JSON responses, but leave the predictor serializer as the default (which for PyTorch isNumpySerializer). As a result, the payload passed to the endpoint is not actually JPEG file data, but a JPEG byte array packaged in a NumPy array.This functionally works, but it means that the data on the wire is kind of an awkward format: If you deploy the endpoint as Async, the object that gets saved to S3 is not actually a JPEG/PNG/etc image, but something that needs a bit of magic to open. Likewise if you set up a real-time endpoint with data capture, the format of the captured data is not ideal.
Suggested updates
I suggest updating the notebooks to use a raw data serializer such as the
DataSerializer(which can accept either data or filenames), which will also require slightly tweaking the inference.pyinput_fncode to parse the image data as expected.