Running instructions to Recreate WNED-CWEB and ACQUAINT results

[danishebadulla326]

I am trying to recreate the results from the paper. I was able to get all the ZESHEL experiment results but I was not able to find any code to get the WNED-CWEB and ACQUAINT results. Could you please give the steps to be followed to recreate these results?

[wutaiqiang]

DATA

In the data folder, we provide the datasets in "benchmark", you need to change the corresponding args to run.

The data processing would take a while as the document is huge. [process via load_entity_description function in data_loader.py ]

MODEL

Following BLINK, for Wikipedia-based datasets, we pre-train the BERT-large on 9M paired Wikipedia samples. Hence, the results for baseline and GER are high. To get the pre-trained BERT-large, please download the ckpt via download_blink_models.sh in https://github.com/facebookresearch/BLINK and convert it into GER format.

[danishebadulla326]

What are the recommended split sizes for the two datasets in data_loader.py? Currently the WORLDS dictionary only has information about zeshel dataset

WORLDS = { 'train': [("american_football", 31929), ("doctor_who", 40281), ("fallout", 16992), ("final_fantasy", 14044), ("military", 104520), ("pro_wrestling", 10133), ("starwars", 87056), ("world_of_warcraft", 27677)], 'valid': [("coronation_street", 17809), ("muppets", 21344), ("ice_hockey", 28684), ("elder_scrolls", 21712)], 'test': [("forgotten_realms", 15603), ("lego", 10076), ("star_trek", 34430), ("yugioh", 10031)] }

I understand that in the paper the pretrained model used was the BERT-large from BLINK, but can I run the experiments with bert-base-uncased as well without any additional changes?

[wutaiqiang]

Following BLINK, we train on AIDA train set and test on WNED and AQUAINT.

[wutaiqiang]

I understand that in the paper the pretrained model used was the BERT-large from BLINK, but can I run the experiments with bert-base-uncased as well without any additional changes?

you can change the backbone to bert-base-uncased by changing the args and no need to change the code.