-
Hi.
I am attempting to get bark set up on my local machine.
Here are the following steps that I have done:
1. Created an virtual environment and activated it
2. I've installed `transformer…
-
### System Info
container built on 24th of october:
including
pip install tqdm
pip install torch
pip install torchvision
pip install transformers
pip install deepspeed==0.15.2
pip…
-
Evidence:
We finetune without Unsloth in Qlora with rank 32, targets all linear layers AND embed/lm_head (smaller 10x lr, same padđing right key as Unsloth) in a total of **1,134,559,232 trainable …
-
**Description of the issue**
The `optimize_for_target_gateset` crashes when trying to optimize XXPowGate with larger global shift value. The optimizer crashes with division by zero error.
**How …
-
I want to add a easy to medium question on NLP category related to Transformers positional encoding upon the input embeddings. The question will be done using only `tensorflow as tf, tf.cast(), tf.con…
-
### Python -VV
```shell
(codestral) ➜ dev python -VV
Python 3.10.14 (main, May 6 2024, 19:42:50) [GCC 11.2.0]
```
### Pip Freeze
```shell
(codestral) ➜ dev pip freeze
absl-py==2.1.0
addict==…
-
Hi all, just a heads up: I filed an [issue](https://github.com/huggingface/transformers/issues/29466) with `huggingface/transformers` requesting model support for BASED via their library.
My engage…
-
How can we take advantage of https://huggingface.co/docs/transformers/transformers_agents ?
-
我的需求是希望能够做专用的小模型（纯个人使用），但小模型需要具备基本的逻辑能力，并且个人没太多资源，顶多两张4090。
所以，我想通过看看是否能够通过蒸馏的方法解决这个需求。
这是一篇将transformers架构通过蒸馏的方法转换为ssm
[Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Mo…
-
### System Info / 系統信息
transformers 4.46.0，python3.10
### Who can help? / 谁可以帮助到您？
_No response_
### Information / 问题信息
- [ ] The official example scripts / 官方的示例脚本
- [X] My own modified scripts …