facebookresearch / PyTorch-BigGraph

Generate embeddings from large-scale graph-structured data.
https://torchbiggraph.readthedocs.io/
Other
3.37k stars 449 forks source link

Regarding negative batch sampling in dyamic relations #256

Open sachin040404 opened 2 years ago

sachin040404 commented 2 years ago

In the negative batch sampling for dynamic relations (url, where it says:

In case of dynamic relations the batch contains edges of the form (𝑥𝑖,𝑟𝑖,𝑦𝑖), with possibly a different 𝑟𝑖 for each 𝑖. If negative sampling and operator application worked the same, it might end up being necessary to transform each right-hand side entity multiple times in several ways, once for each different relation type of the edges the entity is a negative for. This would multiply the number of required operations by a significant factor and cause a sensible performance hit.

To counter this, operators are applied differently in case of dynamic relations. They are applied to either the left- or the right-hand side (never both at the same time), and a different set of parameters is used in each of these two cases. On an input edge (𝑥𝑖,𝑟𝑖,𝑦𝑖) both ways of applying the operators are performed (separately). For the negatives of the form (𝑥′𝑖,𝑗,𝑟𝑖,𝑦𝑖) (i.e., with the left-hand side entity changed), the operator is only applied to the right-hand side. Symmetrically, on (𝑥𝑖,𝑟𝑖,𝑦′𝑖,𝑗), the operator is only applied to the left-hand side. This means that the operator is ever only applied to the entities of the original positive input edge, not on the entities of the negatives. Thus the number of operator evaluations is equal to the number of input edges in the batch.

======================= I have the following question: It seems for same-batch negative sampling for dynamic relations, it will be having negative edges for the entities which were not supposed to have any relations in the first place. For example, if a graph is having only two relations (buys and manufacuredBy) like user,buys,proudct and product,manufacuredBy,producder. But due to dynamic relations, it might have the negative edges involving user and producer which would have been never in the first place there if it had been non-dynamic relation same batch negative sampling (as we first filter all the edges based on the same relation) ?

Did i miss something ?

sachin040404 commented 2 years ago

Just to add: for user and producer there should be no negative or positive edges at all, but in dynamic relations same batch negative sampling, we might have some negative edges involving the user and producer ?

sachin040404 commented 2 years ago

@lw Please let me know if you need any further information ?