[Feature] application/x-npz (numpy archive) deserialization support

[athewsey]

Describe the feature you'd like

npz format is a native serialization option in numpy that can save multiple arrays in a single object via np.savez_compressed(stream, arr1=..., arr2=..., ...))

Although .npz data can be loaded just like .npy (with np.load(obj)), SageMaker's deserializers.NumpyDeserializer currently explicitly checks returned content type from the endpoint, and therefore cannot be used with npz archives even if users set the accept argument in the constructor.

We should update the provided deserializer to enable using this format easily, allowing user's endpoints to return multiple array objects in a native numpy way if they want.

How would this feature be used? Please describe.

From an endpoint, users would return multiple objects compressed together similarly to the below (SKLearn script mode) example code:

def output_fn(prediction, content_type):
    scores, confidences, explains = prediction
    if content_type == "application/json":
        return {"scores": scores, "conf": confidences, "explains": explains}
    elif content_type == "application/x-npz":
        buffer = BytesIO()
        np.savez_compressed(buffer, scores=scores, conf=confidences, explains=explains)
        return buffer.getvalue()

On the client side, users would instantiate the predictor with deserializer configured for npz data:

predictor = sagemaker.Predictor(
    "endpoint_name",
    serializer=sagemaker.serializers.NumpySerializer(),
    deserializer=sagemaker.deserializers.NumpyDeserializer(accept="application/x-npz"),
)

...The endpoint would be able to return, and the client side un-pack, multiple arrays compressed together:

result = predictor.predict(mydata)
result["explains"]

As of today, the above would fail with ValueError "cannot read content type application/x-npz" from the NumpyDeserializer class.

Describe alternatives you've considered

• A separate deserializer for npz data seems unnecessary as it's a less common format, and the same np.load() function used in the current implementation can already read npz data just fine
• Leaving the SDK as-is and expecting users to implement their own deserializer to handle npz format if needed: Again, this seems unnecessary when the current class already seems to offer support except for the explicit MIME type check.

Additional context

N/A

[david-waterworth]

This would be useful. As a workaround I've been been using the NumpySerializer to send a single X ndarry to my model. The model produces multiple outputs (probabilities, labels, embeddings) which I return as

np.savez_compressed(buffer, probabilities=probabilities, labels=labels, embeddings=embeddings), "application/x-npy"

On the client I use the existing sagemaker.deserializers.NumpyDeserializer which works fine, but it returns a numpy.lib.npyio.NpzFile which is dict-like. I then manually extract the contains, i.e.

response = predictor.predict(batch)
probabilities, embeddings, labels = response["probabilities"], response["embeddings"], response["labels"]

Given how well this works it would be nice if sagemaker.deserializers.NumpyDeserializer could handle application/x-npz in a similar format - i.e. after calling np.load it should convert the NpzFile to a tuple

[athewsey]

I believe this is fixed by the merge of the linked PR - closing the issue