Open philschmid opened 8 months ago
What is the value of s3_model_uri
on this line?
model_data={'S3DataSource':{'S3Uri': s3_model_uri + "/",'S3DataType': 'S3Prefix','CompressionType': 'None'}},
I tried with s3://mybucket/neuronx/sdxl/
and s3://mybucket/neuronx/sdxl
. The strucutre is as shown in the image.
Here is a full example https://github.com/philschmid/huggingface-inferentia2-samples/blob/main/stable-diffusion-xl/sagemaker-notebook.ipynb
You just need to change the "3. Upload the neuron model and inference script to Amazon S3" section and then "4. Deploy a Real-time Inference Endpoint on Amazon SageMaker"
Hi @philschmid, I tried your repo but can not reproduce the issue. Does the instance_type
matter?
I don't develop the SDK but i tested with inf2.xlarge
maybe there is something different.
Could you test your code with other instance types?
The error is with inf2.xlarge
, the instance i want to use to deploy a model. Thats where the error appears. Why do you want to test another one?
I want to confirm whether the issue is in the SDK logic or in another place.
@trungleduc, I understand you are trying to troubleshoot the root cause of the issue, but asking me to test on other instance types doesn't seem helpful at this point.
As I mentioned, the error only occurs on inf2.xlarge with the version i shared. It would be more productive to dig deeper into what specifically is failing on inf2.xlarge, where this /
gets added.
Describe the bug SageMaker adds wrongly![image](https://github.com/aws/sagemaker-python-sdk/assets/32632186/341c3ed2-1954-4fd1-923f-878adb63a78b)
/
when usingS3DataSource
where files are stored in an nested order, see screenshot of how my s3 directory looks.To reproduce
S3DataSource
, e.g. belowExpected behavior Deployed endpoint
Screenshots or logs Error:
UnexpectedStatusException: Error hosting endpoint huggingface-pytorch-inference-neuronx-2023-11-07-14-07-46-274: Failed. Reason: error: Key of model data S3 object 's3://sagemaker-us-east-2-558105141721/neuronx/sdxl//text_encoder/model.neuron' maps to invalid local file path..
System information A description of your system. Please provide:
Additional context Add any other context about the problem here.