How can I fine-tune on my own data?

[Joerg99]

Hey, is it possible to fine-tune on another dataset? I fed in my own data with different labels. But this causes a mismatch of tensors. Of course, I could match the labels (it's just naming) but that's not how it should work, right?

[kyzhouhzau]

Yes, no problem was found when i applied it on some different data,. Maybe you should check the codes.

[Joerg99]

Ok, I'll check that. Also, changing to the multilingual model does not work. It's basically the same problem of not matching tensors:

InvalidArgumentError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a mismatch between the current graph and the graph from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error: Assign requires shapes of both tensors to match. lhs shape= [119547,768] rhs shape= [28996,768]

The different shapes are coming from differenct vocabulary sizes (119547 words in multilingual, 28996 in the english model).

I run the code this command:

!/usr/bin/bash

python BERT_NER.py\ --task_name="NER" \ --do_lower_case=False \ --crf=True \ --do_train=True \ --do_predict=True \ --do_eval=True \ --data_dir=data/german \ --vocab_file=cased_L-12_H-768_A-12/vocab.txt \ --bert_config_file=cased_L-12_H-768_A-12/bert_config.json \ --init_checkpoint=cased_L-12_H-768_A-12/bert_model.ckpt \ --output_dir=output/result_dir \ --max_seq_length=128 \ --train_batch_size=32 \ --learning_rate=2e-5 \ --num_train_epochs=4.0

[kyzhouhzau]

I think this problem may caused by different languages. For german, you should download German-based model that released by google.