-
With #1873 and #1874 we implemented loss functions and data augmentation for knowledge distillation based on the TinyBERT paper. This issue here is about training distilled models with that technique …
-
Hi, in your codes for GAD and GAD++, I have 2 questions.
1) GAD and GAD++ does not support setting batchsize>1 in inference.py
2) when I set batchsize=1, strategy="block" is much slower than strateg…
-
## Environment info
- `transformers` version: 4.18.0
- Platform: Linux-3.10.0-1160.59.1.el7.x86_64-x86_64-with-glibc2.17
- Python version: 3.9.12
- Huggingface_hub version: 0.5.1
- PyTorch ve…
-
* Models: base-multilingual-cased, 6 and 3 layers. MaskedLM models.
* Data: mBERT tokenized data for language modeling, using Swedish Gigacorpus or wikitext-2-raw-v1.
* Error: "str object has no a…
-
## Environment info
- `transformers` version: 4.6.1
- Platform: Ubuntu Linux
- Python version:3.88
- PyTorch version (GPU?):1.8.1
- Tensorflow version (GPU?):N/A
- Using GPU in script?:No
-…
-
Run https://github.com/intel/neural-compressor/tree/master/examples/pytorch/nlp/huggingface_models/question-answering/optimization_pipeline/prune_once_for_all/fx script stage:
# for stage 1
python r…
-
## Environment info
- `transformers` version: 4.17.0
- Platform: Linux
- Python version: 3.9.4
- PyTorch version (GPU?): 1.10.2
- Using GPU in script?: Yes
- Using distributed or parallel se…
-
Hi,
Is there any plan to also release the learned cluster centers of the FCN? One would think they are a part of the model. I understand they are not required for inference but may be handy for fine …
-
## Environment info
- `transformers` version: 4.17.0
- Platform: ubuntu18.04
- Python version: 3.6.9
- PyTorch version (GPU?): 1.10.0 GPU
- Tensorflow version (GPU?): none
- Using GPU in scr…
-
## Environment info
- `transformers` version: 4.17.0
- Platform: Linux-4.14.105-1-tlinux3-0013-x86_64-with-glibc2.17
- Python version: 3.9.2
- PyTorch version (GPU?): 1.10.2+cu113 (True)
- Te…