Baselines

[lujiaying]

TransE

• Random initialized token embedding, token vocabulary include every token in train set.
• LSTM as composition functions to obtain mention/relation embedding from token embedding
• h+r~=t as score function
• CGC evaluation: MAP(top15 at most), MRR, Hits@1,3,5.

Done

TransE not work for OLP task. Taxo-TransE also not helpful. MSCG contains too dense Taxo edges, think about other ways for ego graph construction.

[lujiaying]

Jul 22 Performance is really bad in OLP task. Thinking about some mechanism to add our learn-to-generalize idea upon TransE

TODO

• [x] backup dataset
• [x] Glove to init
• [x] mention/relation token overlap analysis. Re-sample the OLP dataset for reasonable performance.
• [x] Add freq threshold for SEMdical, SEMusic x OPIEC,ReVerb; others waiting to finish

[lujiaying]

ConvE

• Inverse relation embedding for training, evaluating (?, r, t)

Details see #20

[lujiaying]

TransH

https://github.com/thunlp/OpenKE/blob/OpenKE-PyTorch/openke/module/model/TransH.py

[lujiaying]

R-GCN

Details see #19

[lujiaying]

CompGCN

Tricks:

• corrupted head, tail batches for head/tail prediction
• 1 v.s. N BCE training loss
• Label smoothing
• Since it is under a transductive setting, put every node(train,dev,test set) in the graph. Same for edges

TODO

• [x] how to deal with big graph? It takes three days to get the result.

  INFO - main - [Wed Aug 11 11:40:57 2021] #node_vocab=51937, #edge_vocab=1428, #ment_vocab=5833, #cep_vocab=5293
  INFO - main - DataLoader Done, trt #triple=381101, graph info:Graph(num_nodes=51937, num_edges=380008,
    ndata_schemes={}
    edata_schemes={'e_vid': Scheme(shape=(1,), dtype=torch.int64)})
  INFO - main - CGC dev #triple=4514, test #triple=15800
  INFO - main - OLP dev #triple=12870, test #triple=38605

[lujiaying]

Count-Infer

Infer relation from both parent and child: get 0 scores too

Infer taxonomy from relation: get 0 scores since edges are rarely shared.

---

Chang into infer by semantic edges? Done c0e5f78