-
Hello,
I want to report and ask for suggestions regarding the quantization of CodeLlama 7B with AQLM.
I followed the instruction proposed in the README file, and quantized this model with the 1x15 g…
-
### System Info
```shell
Debian 11 on CPU, Python3.10
optimum : 1.13.1
onnx : 1.14.1
onnxruntime : 1.15.1
```
### Who can help?
@philschmid, @michaelbenayoun, @JingyaHuang, @echarlaix
### Info…
gidzr updated
11 months ago
-
what is the base-PT models of sqlcoder-7b-2?
codellama or starcode?or others?
Thanks a lot
OKC13 updated
3 months ago
-
The gpu memory usage continues to increase after each round while finetuning LLM with an adapter. The gpu memory increment after each round was approximately the same. I speculate it's because that th…
-
Hi, there first of all this is an amazing project which you have built
But it seems that for each query even the basic ones token usage is in 40-50k
fopr queries based on personal .csv data (.csv…
-
Please feel free to discuss in this thread anything about the InfiCoder-Eval evaluation framework. We welcome any feedback and comments!
-
**Describe the bug**
I encountered an issue when using DeepSpeed 0.12.4 with the [OpenChat trainer](https://github.com/imoneoi/openchat), where checkpointing failed and raised an NCCL error. However,…
-
### System Info
the codellama model from hugging face :(https://huggingface.co/codellama/CodeLlama-7b-Instruct-hf) is Peft Lora tuned to have an adapter.
So i have 2 versions, one where i add th…
-
Thanks for this amazing project.
I'm using OpenAI API extension from the **oobabooga web UI** [https://github.com/oobabooga/text-generation-webui/tree/main/extensions/openai](url)
Config:
OPENA…
-
### Issue you'd like to raise.
I have installed langchain and ctransformer using -
```
pip install langchain
pip install ctransformers[cuda]
```
I am trying following piece of code -
```
…