Open vincentclaes opened 11 months ago
@mohanasudhan Is it ok if I have a look at this issue and propose a PR? If you have any existing code, that can be used as inspiration for me let me know!
@vincentclaes Please feel free to propose a PR. I don't have a specific code sample; however, consider looking at the new ModelBuilder
class that has been created to simplify the deployment.
https://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-modelbuilder-creation.html
Describe the feature you'd like
Being able to deploy huggingface multimodal models to a sagemaker endpoint. Currently only language models are supported that require a prompt as input. Multimodal models like Llava / CLIP / ... require a prompt and an image as input and this is currently not supported.
How would this feature be used? Please describe.
This is how the feature will be used by the end user:
Describe alternatives you've considered
You can package the model yourself and provide an inference.py script, but you have to download the model and tar.gz which takes a lot of time.
Additional context
I came up with this idea when I created a tar.gz for llava with an inference.py and made it available to the world. See my LinkedIn post here: https://www.linkedin.com/posts/vincent-claes-0b346337_aws-sagemaker-huggingface-activity-7141776348963885056-Uv0g?utm_source=share&utm_medium=member_desktop