-
### Your current environment
python3 python -m vllm.entrypoints.api_server --model TheBloke/CodeLlama-7B-Python-AWQ --quantization awq
### How would you like to use vllm
I want to host offline envi…
-
Hi,
Apologies if the solution is obvious but I'm new to this. When running the example infilling script:
`torchrun --nproc_per_node 1 example_infilling.py --ckpt_dir CodeLlama-7b/ --tokenizer_pat…
dv347 updated
2 weeks ago
-
Is it possible to merge multimodal LLMs?
For example, could Llava and CodeLlama be merged? It might be beneficial for some software engineering tasks
-
I'm keeping https://github.com/ErikBjare/are-copilots-local-yet up-to-date, and would love to see some codellama numbers given it's now SOTA :)
-
from #13: let's add a `--bootstrap` flag for setup of the initial state of fake GitHub API
-
Has anyone gotten 16k context length with codellama or llama2? because i have tried multiple models but they all start producing gibberish when the context window gets past 4096. I am using exllama an…
-
### Describe the bug
config.yaml file: llm.model: "huggingface/codellama/CodeLlama-34b-Instruct-hf"
Model: openai/huggingface/codellama/CodeLlama-34b-Instruct-hf
This is from config.yaml file:
…
-
It might be that I just can't find the right setting to make this work, but CodeLlama's upstream model docs refer to a [fill_token](https://huggingface.co/docs/transformers/main/model_doc/code_llama#t…
-
### Before submitting your bug report
- [X] I believe this is a bug. I'll try to join the [Continue Discord](https://discord.gg/NWtdYexhMs) for questions
- [X] I'm not able to find an [open issue](ht…
-
### Describe the bug
I am trying to use the WebSurferAgent as described here https://nbviewer.org/github/microsoft/autogen/blob/main/notebook/agentchat_surfer.ipynb
Here is my code:
```python
…