Closed Vfgandara closed 2 years ago
There's not a good way to freeze Transformers at the moment. You can set grad_factor = 0
to disable weight updates, but computation will still be performed. We are working on improving this.
I'll try the grad_factor = 0
then! Thanks for the help :)
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
How to reproduce the behaviour
Hi there,
I'm trying to train two BERT models in Portuguese with a NER head. The BERT models are BERTimbau and biobertpt-all. To do the training I'm creating a docker container with the following Dockerfile:
Inside data there are the train and dev
.spacy
files for training and the.cfg
bellow:It's dentrimental to the project I' working on that the
transformer
pipe is frozen. But when I runpython3.8 -m spacy train a.cfg -o foo -g 0 --paths.train train.spacy --paths.dev test.spacy
inside of the container I get the following error:If I don't freeze the transformer component things run as usual, no problem. Changing from GPU to CPU also didn't solve the error.
I didn't find much or anything that helpful online so I'm asking here whether there's something that could be done to solve this. Thanks in advance!
Your Environment