search

THUDM / GATNE

Source code and dataset for KDD 2019 paper "Representation Learning for Attributed Multiplex Heterogeneous Network"
MIT License
525 stars 141 forks source link

question on negative sampling and equation 13 #31

Closed Refrainlhy closed 4 years ago

Refrainlhy commented 4 years ago

hi, It is a very solid paper, but I meet some questions... could you give me some hints?

I see you use NCE loss in the code, however, there are many types of nodes, but NCE loss does not consider the type of negative sampling. Hence, during the negative sampling, you do not consider the type of nodes. Am I correct?

And another thing is that one context may have different words. When doing negative sampling, we only give the sampler one true label, so how the sampler prevent itself from sampling true labels? May this point not related to this work, but could you give me some hints? I can not find any materials on the website regarding this question.

The last question is about the equation 13, I see you use the D_z on the features of the target node. Since there is a h_z focusing on the features of the target node, so is the D_z necessary? Do you do the experiment without D_z? is the performance not so good?

Thank you very much!

cenyk1230 commented 4 years ago

Hi @Refrainlhy,

Thank you for pointing these out.

1) Our work mainly focuses on the multiplex part and we do not consider the node types in the code. For node heterogeneity, we simply follow metapath2vec[1] to use the heterogeneous skip-gram. As you can see, the three datasets (Amazon, Twitter, YouTube) used by our work have only one node type.

2) It doesn't matter when other true labels are sampled from the sampler. From the loss function, we don't need to guarantee this. Intuitively, the negative sampling is used to approximate the full softmax. Maybe sampled softmax loss is a better choice and at least the name is not ambiguous.

3) Someone has pointed this out before. The D_z term appears in equation 13 because of some reason and seems unnecessary now.

Refrainlhy commented 4 years ago

I get it！ Thank you for so quick reply.