search

AnacletoLAB / grape

🍇 GRAPE is a Rust/Python Graph Representation Learning library for Predictions and Evaluations
MIT License
502 stars 38 forks source link

Question about Node2VecSkipGramEnsmallen #59

Closed chunyuma closed 3 months ago

chunyuma commented 4 months ago

Hello, I am using Node2VecSkipGramEnsmallen from grape.embedders to generate node embeddings for the Cora graph. Please see my code below:

from grape.datasets.linqs import Cora
from grape.embedders import Node2VecSkipGramEnsmallen
graph = Cora(cache_path="lings_graphs")

embedding = Node2VecSkipGramEnsmallen(embedding_size=100,
                                      epochs=200).fit_transform(graph)

After training is done, I got two embedding matrices for the nodes. Could you please help me explain why there are two embedding matrices? What are the differences between them? I assume it should have only one embedding for each node. Thanks!

LucaCappelletti94 commented 4 months ago

Hi! SkipGram & CBOW models produce two embeddings, one associated to central tokens, one associated to contextual tokens. Commonly, SkipGram consider its embedding the central tokens one, and CBOW the contextual tokens one. Since this selection is arguably arbitrary, you get to choose which one to use (or both at once).

In SkipGram the first returned is the central and the latter the context, if I recall correctly.

chunyuma commented 3 months ago

I see. Thanks so much for the responses. @LucaCappelletti94