Closed michaelgloeckner closed 11 months ago
Create Llama2Service which uses llama2 7b endpoint of kubernetes or sagemaker.
Kubernetes example call: curl -X POST -H 'Content-Type: application/json' -d '{ "messages": [{"role": "user", "content": "How are you?"}], "model": "llama-2-7b-chat.ggmlv3.q4_0.bin", "stream": false}' http://genai.one-cx.org:8000/v1/chat/completions
Sagemaker
Create Llama2Service which uses llama2 7b endpoint of kubernetes or sagemaker.
Kubernetes example call: curl -X POST -H 'Content-Type: application/json' -d '{ "messages": [{"role": "user", "content": "How are you?"}], "model": "llama-2-7b-chat.ggmlv3.q4_0.bin", "stream": false}' http://genai.one-cx.org:8000/v1/chat/completions
Sagemaker