Error in subword pooling.

[geo47]

Hi @dsindex

Getting following error when using --bert_use_subword_pooling:

ntagger/model.py", line 737, in forward
    embed_out = torch.cat([embed_out, pos_embed_out], dim=-1)
RuntimeError: Sizes of tensors must match except in dimension 1. Got 180 and 0 (The offending index is 0)

[dsindex]

did you set —bert_use_subword_pooling option for preprocess.py?

i guess x[3] of input x (for pos_embed_out) or x[5](for word2token_idx) was not properly set.

and the line 737 indicates different code in the recent code base. https://github.com/dsindex/ntagger/blob/master/model.py

what about update source and re-run preprocess.py, train.py?

[geo47]

Yes, I did set —bert_use_subword_pooling option for preprocess.py and also for train.py.

The line is different because i disabled the DSA pooling and using mean of last four layers. I am not sure if the problem occurs due to not using DSA pooling.

The problems occurred due to embed_out of the use_subword_pooling.

[dsindex]

ok! i fixed it! some bugs for dataset generation. plz update util_bert.py and then run again:

$ python preprocess.py --config configs/config-bert.json --data_dir=data/conll2003 --bert_model_name_or_path embeddings/BERT-Finetuned --bert_use_subword_pooling

$ python train.py --batch_size 64 --eval_batch_size 128 --config configs/config-bert.json --data_dir data/conll2003 --bert_model_name_or_path embeddings/BERT-Finetuned --save_path pytorch-model-bert-en.pt --bert_output_dir bert-checkpoint-en --epoch 30 --bert_use_pos --use_char_cnn --use_mha --bert_use_feature_based --use_crf --bert_use_subword_pooling

[geo47]

Hi @dsindex , It is fixed now. :+1: However, it degraded the model performance with a significant margin. Thanks.