[Solutions] 1st

[riow1983]

[Discussion] 1st place solution: Metric learning and GPT

1st solution: Matching the : Context Similarity via Deep Metric Learning and Beyond

[Code]

• submit gpt spacy
• biomed_roberta+scibert-base

[GitHub]

• https://github.com/suicao/coleridge-gpt

[一言で言うと] 以下２アプローチをそれぞれ別個に用意:

• GPT(QAタスク) + beamsearch (private LB = 0.565 (0.594 w/o labels obtained by scispaCy's abbreviation detecotor)) by Khoi Nguyen
• metric learning with usual MLM backbones (private LB = 0.576 (0.588 w/o labels obtained by scispaCy's abbreviation detecotor)) by Nguyen Quan Anh Minh
  

---

[関連知識]

• Source code for pytorch_transformers.modeling_gpt2
• torch.utils.data.TensorDataset https://schemer1341.hatenablog.com/entry/2019/01/06/024605
• Is it okay to create a Loss function within the forward method?
• ArcFace loss モダンな深層距離学習 (deep metric learning) 手法: SphereFace, CosFace, ArcFace

[riow1983]

GPT + beamsearchについては, データセットの言及そのものではなく, データセットが言及される文脈を学習させる手法であると解釈できる:

Very nice way to force model learn the context of dataset mentions rather than the mentions themselves. I thought this was the problem to solve but did not thought of your solution. Congrats on the win and GM title to come.

https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/248251#1361888