Deryuu
commented
1 year ago I managed to solve my former problem, now i get following error:
Traceback (most recent call last):
File "train.py", line 326, in <module>
main()
File "train.py", line 288, in main
logging_output = trainer.train_step(samples)
File "/pfs/data5/home/ma/ma_ma/ma_tuvu/workspace/tabert/utils/trainer.py", line 134, in train_step
total_loss, logging_output = self.model(**sample)
File "/home/ma/ma_ma/ma_tuvu/miniconda3/envs/tabert/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
result = self.forward(*input, **kwargs)
File "/pfs/data5/home/ma/ma_ma/ma_tuvu/workspace/tabert/table_bert/vertical/vertical_attention_table_bert.py", line 341, in forward
masked_cell_token_loss = loss_fct(cell_token_scores.view(-1, self.config.vocab_size), masked_cell_token_labels.view(-1))
File "/home/ma/ma_ma/ma_tuvu/miniconda3/envs/tabert/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
result = self.forward(*input, **kwargs)
File "/home/ma/ma_ma/ma_tuvu/miniconda3/envs/tabert/lib/python3.7/site-packages/torch/nn/modules/loss.py", line 916, in forward
ignore_index=self.ignore_index, reduction=self.reduction)
File "/home/ma/ma_ma/ma_tuvu/miniconda3/envs/tabert/lib/python3.7/site-packages/torch/nn/functional.py", line 2009, in cross_entropy
return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
File "/home/ma/ma_ma/ma_tuvu/miniconda3/envs/tabert/lib/python3.7/site-packages/torch/nn/functional.py", line 1838, in nll_loss
ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
RuntimeError: invalid argument 2: non-empty vector or matrix expected at /tmp/pip-req-build-4baxydiv/aten/src/THCUNN/generic/ClassNLLCriterion.cu:31Hope someone can help me with this issue.
Hey, i try to run following:
python train.py \--task vertical_attention \--data-dir data/train_data/vertical_tabert \--output-dir data/runs/vertical_tabert \--table-bert-extra-config '{"base_model_name": "bert-base-uncased", "num_vertical_attention_heads": 6, "num_vertical_layers": 3, "predict_cell_tokens": true}' \--train-batch-size 8 \--gradient-accumulation-steps 64 \--learning-rate 4e-5 \--max-epoch 10 \--adam-eps 1e-08 \--weight-decay 0.01 \--fp16 \--clip-norm 1.0 \--empty-cache-freq 128and if "predict_cell_tokens" is set true i get following error
Traceback (most recent call last):'File "train.py", line 326, in <module> main()'File "train.py", line 288, in main logging_output = trainer.train_step(samples)File "/pfs/data5/home/ma/ma_ma/ma_tuvu/workspace/tabert/utils/trainer.py", line 134, in train_step total_loss, logging_output = self.model(**sample)File "/home/ma/ma_ma/ma_tuvu/miniconda3/envs/tabert/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__ result = self.forward(*input, **kwargs)File "/pfs/data5/home/ma/ma_ma/ma_tuvu/workspace/tabert/table_bert/vertical/vertical_attention_table_bert.py", line 339, in forward cell_token_scores = self.span_based_prediction(masked_cell_representation, masked_cell_token_position_embedding)File "/home/ma/ma_ma/ma_tuvu/miniconda3/envs/tabert/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__ result = self.forward(*input, **kwargs)File "/pfs/data5/home/ma/ma_ma/ma_tuvu/workspace/tabert/table_bert/vertical/vertical_attention_table_bert.py", line 157, in forward self.dense2(h)File "/pfs/data5/home/ma/ma_ma/ma_tuvu/workspace/tabert/src/pytorch-pretrained-bert/pytorch_pretrained_bert/modeling.py", line 124, in gelu return x * 0.5 * (1.0 + torch.erf(x / math.sqrt(2.0)))RuntimeError: cuda runtime error (9) : invalid configuration argument at /opt/conda/conda-bld/pytorch_1579022060824/work/aten/src/THC/generic/THCTensorMathPointwise.cu:182when "predict_cell_tokens" is set false, it works fine. I hope you can help me and thank you guys in advance.