search

allenai / commonsense-kg-completion

MIT License
105 stars 16 forks source link

fine-tune bert #16

Open Jiang-X-Pro opened 2 years ago

Jiang-X-Pro commented 2 years ago

thanks for your great work i feel confused about the way to fine-tuned the bert as mentioned before, the model remove the relation, so the input text is {head entity token} concat {tail entity token} ?

Jiang-X-Pro commented 2 years ago

how can I fine-tune the bert with text of FB15K-237 should I choose the file "FB15K-237/train.txt"?

chaitanyamalaviya commented 2 years ago

Hi, the input for finetuning BERT was the phrases representing the nodes (not the edges/triples themselves). Since commonsense KGs have natural language phrases as nodes, it made sense to do that. But since FB15k-237 has single tokens as nodes it might make more sense to finetune on relation phrase , where the relation phrase is a natural language phrase representing the Freebase relation.

Jiang-X-Pro commented 2 years ago

Thanks for your reply. Which file should I choose for finetuning Bert among the files you give? or maybe I need to construct a new file by following your tips?

chaitanyamalaviya commented 2 years ago

Hey, yea so since you mentioned you are interested in training models on FB15k-237, you could get that dataset from this repo: https://github.com/TimDettmers/ConvE.