Closed kkangert closed 1 year ago
@kangerts We didn't try this model. But you should be able to try it out with our DJLServing: https://github.com/deepjavalibrary/djl-serving
Here is a few deploy python model examples: https://github.com/deepjavalibrary/djl-demo/tree/master/djl-serving/python-mode
And Large language model deployment blogpost:
@kangerts We didn't try this model. But you should be able to try it out with our DJLServing: https://github.com/deepjavalibrary/djl-serving
Here is a few deploy python model examples: https://github.com/deepjavalibrary/djl-demo/tree/master/djl-serving/python-mode
And Large language model deployment blogpost:
- Deploy Falcon-40B with large model inference DLCs on Amazon SageMaker
- Build custom chatbot applications using OpenChatkit models on Amazon SageMaker
- Deploying LLMs On Amazon SageMaker With DJL Serving
- Achieve high performance with lowest cost for generative AI inference using AWS Inferentia2 and AWS Trainium on Amazon SageMaker
- Deploy large models at high performance using FasterTransformer on Amazon SageMaker Okay, thank you.
May I ask if DJL can use ChatGLM-6B( https://github.com/THUDM/ChatGLM-6B )Are you unfamiliar with DJL and would like to use it in company projects? Can you give me some examples?