I am following the following tutorial to do Fine tuning to LiLT in any language, and I would like to know how I can use the nielsr/funsd-layoutlmv3 dataset instead of nielsr/funsd-iob-original on the notebook.
I appreciate any help, thank you in advance!
This is the output error:
../aten/src/ATen/native/cuda/Indexing.cu:1146: indexSelectLargeIndex: block: [567,0,0], thread: [95,0,0] Assertion srcIndex < srcSelectDimSize failed.
File ~/miniconda3/envs/tf/lib/python3.9/site-packages/transformers/trainer.py:1890, in Trainer._inner_training_loop(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval)
1887 self.control = self.callback_handler.on_step_begin(args, self.state, self.control)
1889 with self.accelerator.accumulate(model):
-> 1890 tr_loss_step = self.training_step(model, inputs)
1892 if (
1893 args.logging_nan_inf_filter
1894 and not is_torch_tpu_available()
1895 and (torch.isnan(tr_loss_step) or torch.isinf(tr_loss_step))
1896 ):
1897 # if loss is nan or inf simply add the average of previous logged losses
...
File ~/miniconda3/envs/tf/lib/python3.9/site-packages/torch/nn/modules/linear.py:114, in Linear.forward(self, input)
113 def forward(self, input: Tensor) -> Tensor:
--> 114 return F.linear(input, self.weight, self.bias)
RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling cublasCreate(handle)
Output is truncated. View as a scrollable element or open in a text editor. Adjust cell output settings...
I am following the following tutorial to do Fine tuning to LiLT in any language, and I would like to know how I can use the
nielsr/funsd-layoutlmv3dataset instead ofnielsr/funsd-iob-originalon the notebook.I appreciate any help, thank you in advance!
This is the output error: ../aten/src/ATen/native/cuda/Indexing.cu:1146: indexSelectLargeIndex: block: [567,0,0], thread: [95,0,0] Assertion
srcIndex < srcSelectDimSizefailed.RuntimeError Traceback (most recent call last) /home/user/folder/test/Fine_tune_LiLT_on_a_custom_dataset,_in_any_language.ipynb Cell 30 line 1 ----> 1 trainer.train()
File ~/miniconda3/envs/tf/lib/python3.9/site-packages/transformers/trainer.py:1589, in Trainer.train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs) 1587 hf_hub_utils.enable_progress_bars() 1588 else: -> 1589 return inner_training_loop( 1590 args=args, 1591 resume_from_checkpoint=resume_from_checkpoint, 1592 trial=trial, 1593 ignore_keys_for_eval=ignore_keys_for_eval, 1594 )
File ~/miniconda3/envs/tf/lib/python3.9/site-packages/transformers/trainer.py:1890, in Trainer._inner_training_loop(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval) 1887 self.control = self.callback_handler.on_step_begin(args, self.state, self.control) 1889 with self.accelerator.accumulate(model): -> 1890 tr_loss_step = self.training_step(model, inputs) 1892 if ( 1893 args.logging_nan_inf_filter 1894 and not is_torch_tpu_available() 1895 and (torch.isnan(tr_loss_step) or torch.isinf(tr_loss_step)) 1896 ): 1897 # if loss is nan or inf simply add the average of previous logged losses ... File ~/miniconda3/envs/tf/lib/python3.9/site-packages/torch/nn/modules/linear.py:114, in Linear.forward(self, input) 113 def forward(self, input: Tensor) -> Tensor: --> 114 return F.linear(input, self.weight, self.bias)
RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling
cublasCreate(handle)Output is truncated. View as a scrollable element or open in a text editor. Adjust cell output settings...