search

lanwuwei / SPM_toolkit

Neural network toolkit for sentence pair modeling.
303 stars 70 forks source link

about STS task #28

Closed BruceLee66 closed 5 years ago

BruceLee66 commented 5 years ago

First,i have not found this function 'readSTSdata()' in your code, Secondly,When the model is initialized, there is a corresponding classification,Such as STS(6), TrecQA(2),but in the end it becomes 2 classes by default in the forward function.Is it really right? Finally,How do you solve the similarity problem with the classified code? Some of the code is not very clear.

lanwuwei commented 5 years ago
  1. Please check this link: https://github.com/lanwuwei/SPM_toolkit/blob/de9bc34d9ad571d85058afdfb496daf7f3817ec1/DecAtt/util.py#L120
  2. For different tasks, I set different value for num_class.
  3. For STS, we should use nn.KLDivLoss(), since it is a regression task. There is label conversion for STS, which you may spend some time understanding it. for line in open(dir + 'sim.txt'): sim = float(line.strip()) ceil = int(math.ceil(sim)) floor = int(math.floor(sim)) tmp = [0, 0, 0, 0, 0, 0] if floor != ceil: tmp[ceil] = sim - floor tmp[floor] = ceil - sim else: tmp[floor] = 1 labels.append(tmp)
BruceLee66 commented 5 years ago

image The code in the image is from the model.py file in the PWIM model.I don't understand why in the STS task, the final dimension of the output is [1, 2].

lanwuwei commented 5 years ago

Ok, I got your point. Actually the PWIM repo is for my NAACL 18 paper, which I tried character embedding for MSRP, URL and PIT. All these tasks have only two classes. Sorry about the confusing part, you can remove this line and change the num_class accordingly.