search

deepjavalibrary / djl-demo

Demo applications showcasing DJL
https://demo.djl.ai
Apache License 2.0
307 stars 127 forks source link

add 01-ai/Yi-34B-Chat-4bits TheBloke/Nous-Hermes-2-Mixtral-8x7B-DPO-A… #431

Closed ghost closed 6 months ago

ghost commented 6 months ago
  1. Add sample code for deploying 01-ai/Yi-34B-Chat-4bits, TheBloke/Nous-Hermes-2-Mixtral-8x7B-DPO-AWQ using vllm backend.
  2. Implement streaming output using SageMaker's invoke_endpoint_with_response_stream.
sindhuvahinis commented 6 months ago

Thank you for the contribution.