Error in using Clarify with AWS Jumpstart image classification models

[zandvakiliramin]

Hi, I have fine-tuned an image classification model using SageMaker jump start.

I have also deployed my model. This is my code for deploying my model:

sgmkr_clnt = boto3.client("sagemaker")

model_id = 'tensorflow-ic-imagenet-nasnet-mobile'
model_s3_key = <path to my model on s3>
model_version = '2.0.2'

model_name = <some name>

response = sgmkr_clnt.create_model(
    ModelName=model_name,
    PrimaryContainer={"Image": deploy_image_uri, "ModelDataUrl": model_s3_key},
    ExecutionRoleArn=SM_ROLE.ARN.value,
)

prefix = <some prefix>
s3_data_input_path = <some address on S3>

analysis_result_path = <some address on S3>

explainability_data_config = clarify.DataConfig(
    s3_data_input_path=s3_data_input_path,
    s3_output_path=analysis_result_path,
    dataset_type="application/x-image",
)

model_config = clarify.ModelConfig(
    model_name=model_name, instance_type="ml.m5.xlarge", instance_count=1, content_type="image/png"
)

predictions_config = clarify.ModelPredictedLabelConfig(label_headers=['NO', 'YES'])

image_config = clarify.ImageConfig(
    model_type="IMAGE_CLASSIFICATION", num_segments=20, segment_compactness=5
)

shap_config = clarify.SHAPConfig(num_samples=500, image_config=image_config)

I have only one img.png as input.

When I run my code I get this error: An error occurred (ModelError) when calling the InvokeEndpoint operation (reached max retries: 0): Received server error (500) from primary with message "{"error": "unsupported content type image/png"}"

I have been able to use this error for Batch Inference using png files, before, but it does not work with Clarify.

[zandvakiliramin]

I found where the problem may be, the following code is the inference.py of sagemaker jumpstart

def input_fn(input_data, content_type):
    """
    Args:
        input_data: the request payload serialized in the content_type format
        content_type: the request content_type
    """
    if content_type == "application/x-image":
        decoded = Image.open(io.BytesIO(input_data))
    else:
        raise ValueError(f"Type [{content_type}] not supported.")

    preprocess = transforms.Compose([transforms.ToTensor()])
    normalized = preprocess(decoded)
    return normalized

Here the only acceptable content_type is "application/x-image" but it is not allowed in ModelConfig. Any idea how to solve this?

[xiaoyi-cheng]

Hi, We consider this as a feature request as here are the current supported content_type. https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/clarify.py#L1170-L1176

I will consult our PM to put this on our roadmap. Thank you!