-
https://github.com/ModelTC/Outlier_Suppression_Plus/blob/3ba97ae2dab0e6e5ead5da1795f50fd47025a49d/quant_transformer/model/quant_llama.py#L242 hi~ i noticed that there are two types LN i.e. pre-LN&pos…
-
Subscribe to this issue and stay notified about new [daily trending repos in C#](https://github.com/trending/c%23?since=daily).
-
In this recent [EXL2 vs GGUF](https://old.reddit.com/r/LocalLLaMA/comments/17w57eu/llm_format_comparisonbenchmark_70b_gguf_vs_exl2/) discussion, one stand-out comment was [this one by llama_in_sunglas…
-
### Checklist
- [X] The issue exists after disabling all extensions
- [X] The issue exists on a clean installation of webui
- [ ] The issue is caused by an extension, but I believe it is caused by a …
-
**Describe the bug**
I have two ubuntu machines, and with 10Gb/s erthnet cable connected and I want to use deepspeed to use these two machines to
run a model training with pipeline parallel, and …
-
### Is there an existing issue for this?
- [X] I have searched the existing issues and checked the recent builds/commits
### What happened?
Autolaunch seems to be enabled by default now, but there …
-
This is the link to HuggingFace model id and Github code demo for running project with onnx model which I convert completed 5 original models
I fixed the Grid Sample 5D error and successfully …
-
### Is there an existing issue for this?
- [X] I have searched the existing issues and checked the recent builds/commits
### What happened?
I installed the stable diffusion web ui and it worked fin…
-
Hello,
I am trying to fine-tune the mixtral-8x22b-instruct model but I keep getting the OOM error.
I am using 3x A100 gpus for a total of 240gb of vram.
I am using QLORA 4bit.
After the first fine…
-
### System Info
```Shell
Python 3.11.5
torch 2.3.0
transformers 4.41.1
accelerate 0.30.1
+-----------------------------…