-
### Is your feature request related to a problem? Please describe.
Hello!
The issue is related to the use of `Together AI` models, such as `CodeLlama-34b` and `Llama-3-70b-chat-hf`.
Despite that …
-
### System Info
Running a TGI 2.0.3 docker on a 8 NVIDIA_L4 VM.
Command:
```bash
MODEL=codellama/CodeLlama-70b-Python-hf
docker run \
-m 320G \
--shm-size=40G \
-e NVIDIA_VISIBLE_DEVIC…
-
Not all computers can run codellama. Being able to select another model would be quite cool for those.
-
Is there a particular dataset format required for finetuning codellama? I have the dataset in the OpenAI suggested format which is basically a jsonl with each entry having messages: [{role: 'system', …
-
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
HuggingFace TGI is a standard way to…
-
Is there going to be an updated version of Codellama based on Meta's new [LLaMa 3 ](https://github.com/meta-llama/)?
-
`macOS 14.1 23B2073`
`node -v`
`v20.12.0`
`npm -v`
`10.5.0`
`tsc -v`
`Version 5.4.3`
I run the installer command:
`git clone https://github.com/pgibler/cmdh.git && cd cmdh && ./install…
-
Hello Guys,
Could you guide me in the right direction to get the configuration of the Code Llama Instruct model right?
I have this config so far:
```
{
"name": "Code Llama",
"e…
-
### Checked other resources
- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation with the integrated search.
- [X] I used the GitHub search to find a…
-
你好,在最近的测试中,我在A100上测试Llama-13b、7b等模型,对比vllm和distserve, 在满足slo的情况下, distserve性能要优于vllm,但是在测试codellama-34b过程中,当我的输入长度为8192,发现TTFT要高出vllm约3倍左右,请问这个情况是正常的吗?vllm使用tp2, distserve使用prefill tp2, decode tp2。