-
🔍 **Problem Description**:
It is a simple project based on predicting the text and quotes that lord god ideas suggest method to overcome our problems in our daily life. It will produce nlp sentences …
-
### What is the issue?
gemma2:9b-instruct-**q6_K** : gemma2:9b-instruct-**q8_0** = **21**t/s : **25**t/s
mistral-nemo:12b-instruct-2407-**q6_K** : mistral-nemo:12b-instruct-2407-**q8_0** = **17**t/s…
-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [ Yes] I am running the latest code. Development is very rapid so there are no tagged versions as…
-
Recent changes in Huggingface Transformers (https://github.com/huggingface/transformers/commit/cdee5285cade176631f4f2ed3193a0ff57132d8b and https://github.com/huggingface/transformers/commit/4a3f1a686…
-
### System Info
`transformers==4.46.1`
`python==3.10.14`
### Who can help?
@muellerzr @SunMarc @ArthurZucker
### Information
- [X] The official example scripts
- [ ] My own modified scripts
##…
-
Thank you for releasing a great project.
I measured [`genai-perf`](https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/perf_analyzer/genai-perf/docs/tutorial.html#profile-g…
-
### 🚀 The feature, motivation and pitch
How to run the int4 quantized version of the gemma2-27b model
### Alternatives
_No response_
### Additional context
_No response_
-
### System Info / 系統信息
Ubuntu 22.04.4 LTS
python 3.10
transformer 4.43.0
cuda 12.0
torch 2.3.0
vllm 0.4.3
### Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？
- [ ] docker / docke…
-
Just opening this to add support for all models following #34184
Lets bring support to all model! 🤗
- [x] Llama
It would be great to add the support for more architectures such as
- [ ] Qwe…
-
Hi @danielhanchen
I am trying to fine-tune gemma2-2b for my task following the guidelines of the continued finetuning in unsloth. Howver, I am facing OOM while doing so. My intent is to train gemm…