-
Hi. I realized that accelerate launch works perfectly when I set batch_size = "auto" but gets stuck at the very end when I use batch_size = "auto:2". The problem persists whether I use evaluator.simpl…
-
**Describe the bug**
When I try to generate a response using a local model the app immediately crashes without error message.
Terminal:
```
Indexing files in directory
config is: { type: 're…
-
Hello,
I've successfully finetuned Llama-3 8B with QDoRA and am now looking to perform inference using vLLM. Could you provide guidance or scripts on how to merge the QDoRA adapters with the origin…
-
Hi guys!
I create MoE models with this [config](https://drive.google.com/file/d/1JwJZCQWZyNRCqgpuxQ-u7qELLxMISPko/view?usp=sharing) and this notebook in [gcolab](https://colab.research.google.com/dri…
-
**Describe the bug**
Attempting to import HuggingFaceTGIGenerator from haystack.components.generators results in an ImportError
**Error message**
ImportError: cannot import name 'HuggingFaceTGIGene…
-
### Your current environment
```text
The output of `python collect_env.py`
Collecting environment information...
PyTorch version: 2.3.1+cu118
Is debug build: False
CUDA used to build PyTorch…
-
### Describe the issue
I'm running the first example from the RAG tutorial [here](https://microsoft.github.io/autogen/blog/2023/10/18/RetrieveChat), using open-source models via LMStudio.
Here is …
-
In the Gemma 7b notebook, when rslora and dora are active, and the settings for 4-bit and 8-bit are off with r=8 and alpha=16, I encounter an error as described below. I have targeted all linear layer…
-
### Reminder
- [X] I have read the README and searched the existing issues.
### System Info
## QWEN2-1.5B(0.5B)
正常
## QWEN2-7B(MoE)
需要使用bf16 #4278
正常
## QWEN2-72B
正常,有一点点问题,只能在8卡上启动(s…
-
I recently came across a model named Zephyr-7b-beta, which seems to be a great fit for my current needs.
I am wondering if Lit-gpt currently supports this model?
Thank you so much!