Erorr occured when I modify real-time inference to batch transform

[isoyaoya]

hello, Thanks for your amazing job~ I followed your code to build end-to-end Hugging Face Transformers with Sagemaker Pipeline. This code use real-time inference and create an endpoint, but I want to achieve batch transform inference.

My code looks like below. ` from sagemaker.huggingface.model import HuggingFaceModel from sagemaker.s3 import S3Uploader,s3_path_join from sagemaker.inputs import TransformInput from sagemaker.workflow.steps import TransformStep

huggingfacemodel = HuggingFaceModel( model_data=step_train.properties.ModelArtifacts.S3ModelArtifacts, # path to your trained sagemaker mod role=role, transformers_version=_transformers_version, pytorch_version=_pytorch_version, py_version=_py_version, sagemaker_session=sagemaker_session, )

output_s3_path = s3_path_join("s3://",'sagemaker-us-east-1-7348*',"y-datalab-modela/output") transformer_instance_type=ParameterString(name="Fp16", default_value="False") transformer = huggingfacemodel.transformer( instance_count=1, instance_type="ml.m5.4xlarge", output_path=output_s3_path, strategy='SingleRecord', )

batch_data = ParameterString( name="BatchData", default_value='s3://sagemaker-us-east-1-73483*/y-datalab-modela/test/stary_data.jsonl', ) transform_step = TransformStep( name="Batch", transformer=transformer, inputs=TransformInput( data=batch_data, content_type='application/json', split_type="Line", ) )

`

but when I run the code, it returns error: ‘AttributeError: 'Properties' object has no attribute 'decode'’.

Could you please help in this issue or could you please provide some sample code about how to use batch transform pipeline? Thank you!

[philschmid]

Could you please format your code to read it better and also share the error you are seeing? And is that the complete code for your example?

[isoyaoya]

Have you tried batch transform inference rather than realtime inference before? Could you please provide some sample codes?

I have attached the whole notebook. Now there are some other errors:

lab_1_sagemaker_pipeline_batchtransform.ipynb.zip

[philschmid]

Could you please add the whole error stack it seems that there is something missing.

[isoyaoya]

log-events-viewer-result (2).csv

[isoyaoya]

Attach the whole error stack, thank you so much!

[philschmid]

From error it seems that you are sending malformed payloads. Here is an example for batch transform: https://github.com/huggingface/notebooks/blob/main/sagemaker/12_batch_transform_inference/sagemaker-notebook.ipynb

[isoyaoya]

I have seen this code before. Of course, if I do not use Sagemaker pipeline, I can create batch transform job without error and output correct predict results. However, I was required to use Sagemaker pipeline and batch transform to achieve automaticlly process.

[isoyaoya]

Have you tried to modify your code from real-time inference to batch transform with Sagemaker pipeline before? In some NLP applications, batch transform is often used.

[isoyaoya]

Use a lambda, create a lambda step to deploy registered model could solve the issue.

[dilky-ascentic]

@isoyaoya can you share the how you configured lambda step to do batch transform please?

[philschmid]

You can find the code here: https://github.com/philschmid/huggingface-sagemaker-workshop-series/tree/main/workshop_3_mlops/utils