search

warchildmd / game2vec

TensorFlow implementation of word2vec applied on https://www.kaggle.com/tamber/steam-video-games dataset, using both CBOW and Skip-gram.
70 stars 6 forks source link

Why use word2vec #1

Open michaelwsherman opened 7 years ago

michaelwsherman commented 7 years ago

I don't understand why you would use word2vec. I think what you want is to learn association rules. Association rules would answer a question like "given someone has played/purchased game x what other games have they played/purchased?" Word2vec is going to be sensitive to the order the games are presented in the data, which is probably not want you want here (seeing as the dataset has no information about what order the games were played/purchased in).

Association rules would not care about order, and would be able to consider the full set of games someone has played/purchased, rather than just the plays/purchases near the game in the possibly arbitrarily ordered data.

Also, if you're training word vectors with a dataset this small, you might want to use gensim--using tensorflow here is like bringing a nuclear missle to a knife fight.

Although maybe I've missed something--why didn't you use association rules? Why use word vectors when you have a set where order doesn't matter, especially with a dataset this small, where you certainly can't expect any problem scaling association rules. Why not use gensim?

warchildmd commented 7 years ago

TBH I wanted to use TF for something + I wanted to have embeddings, that can be used as input to another network.

As for the order, I used bag-of-words and in this approach order didn't matter at all.

michaelwsherman commented 7 years ago

But the order does matter--if the list of the games is bigger than the window size then the games outside the window size aren't part of that data point. Unless you set your window size to be at least as long as the longest list of games.

warchildmd commented 7 years ago

I use a dynamic window. Each user's game is grouped together in a set, so for each user, I have a set of games that is a window.

To create a batch, while I do not have the required amount of (input, output) pairs I do the following:

  1. Randomly select a game set
  2. Using the game set, depending on method (CBOW, Skipgram), generate training pairs.
michaelwsherman commented 7 years ago

Using a dynamic window would introduce some bias, in terms of what games are more likely to be in smaller collections (the most popular games?) are going to be drawn more. Although I'm not positive what the exact effects of this bias would be in terms of the vectors generated.

What's the other network you want to feed these vectors into (and why not have the network generate its own vector representation as part of the training)?