Closed ofrimasad closed 2 years ago
Hey this error seems strange. I believe it is rather pytorch or huggingface transformers related than a problem within FARM.
Have you tried using only the CPU to let the code run?
Actually this post from pytorch says your cuda might be running out of memory, so you should just try to lower the batch size or max_seq length. See https://discuss.pytorch.org/t/cuda-error-cublas-status-not-initialized-when-calling-cublascreate-handle/125450/2
hi @Timoeller.
I have actually managed to narrow this down.
My model expects batch['segment_ids']
to be all 0 (just like Roberta does).
When I use deepset/roberta-base-squad2
model this is exactly what happens.
but when using my model i get batch['segment_ids']
containing 1 as well.
I can't find in the documentation any way to set all segment_ids to 0. I suspect it is looking for the model to be Roberta at some point of your code...
Thanks
This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 21 days if no further activity occurs.
Question Hey I am trying to train a language model called
onlplab/alephbert-base
(a Hebrew language model, closest to Roberta). But when i calltrainer.train()
I get an error:Any idea why is that happening?
My batch size is small (not close to filling the GPU mem). I have tried
LanguageModel.load(lang_model, language_model_class='Roberta')
(since my model also does not use token_type_ids)Thanks
Additional context farm version 0.8.0