Comparison for concatenation vs sum of embeddings

[sdeepaknarayanan]

Hi, First off, very nice and interesting work 👍. In the paper, there's a claim that's made regarding the embedding concatenation v/s sum of all the embeddings.

e. Thus, we change the way of obtaining final embedding from concatenation (i.e., e ∗ u = e (0) u ∥· · · ∥e (L) u ) to
sum (i.e., e ∗ u = e (0) u + · · · + e (L) u ). Note that this change has little effect on NGCF’s performance, but makes the following ablation studies more indicative of the embedding quality refined by GCN.

Is there empirical evidence for this claim?

Thanks

[hexiangnan]

If we concat the embeddings, the representation space is enlarged. But if we sum up the embeddings, the representation space remains the same. That's what we mean by "more indicative of the embedding quality refined by GCN.". Hope it clarifies.

On Tue, Aug 25, 2020 at 12:47 PM Deepak Narayanan notifications@github.com wrote:

Hi, First off, very nice and interesting work 👍. In the paper, there's a claim that's made regarding the embedding concatenation v/s sum of all the embeddings.

e. Thus, we change the way of obtaining final embedding from concatenation (i.e., e ∗ u = e (0) u ∥· · · ∥e (L) u ) to

sum (i.e., e ∗ u = e (0) u + · · · + e (L) u ). Note that this change has little effect on NGCF’s performance, but makes the following ablation studies more indicative of the embedding quality refined by GCN.

Is there empirical evidence for this claim?

Thanks

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kuandeng/LightGCN/issues/8, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAI3DDSD5Z5BN6N5ATJ6CGTSCM647ANCNFSM4QKGXONA .

-- Xiangnan HE (Dr) :: Professor :: Vice Dean, School of Data Science :: University of Science and Technology of China (USTC) :: 443 Huangshan Road, Hefei, China 230027 :: (0551) 63607236 (DID) :: hexn@ustc.edu.cn (E) :: http://staff.ustc.edu.cn/~hexn/ (W)

[sdeepaknarayanan]

Hi @hexiangnan, Thanks for the quick response. I understood your point. However, my question was more along the lines of performance. Did you run experiments to compare concatenation with sum, because the paper seems to claim that this has little effect on the performance. I was hoping to see some experiments on the same.

[hexiangnan]

As I remember, Deng Kuan (the second author) did the experiments. Concating does not degrade the performance, also does not improve.

On Tue, Aug 25, 2020 at 1:57 PM Deepak Narayanan notifications@github.com wrote:

Hi @hexiangnan, Thanks for the quick response. I understood your point. However, my question was more along the lines of performance. Did you run experiments to compare concatenation with sum, because the paper seems to claim that this has little effect on the performance. I was hoping to see some experiments on the same.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

-- Xiangnan HE (Dr) :: Professor :: Vice Dean, School of Data Science :: University of Science and Technology of China (USTC) :: 443 Huangshan Road, Hefei, China 230027 :: (0551) 63607236 (DID) :: hexn@ustc.edu.cn (E) :: http://staff.ustc.edu.cn/~hexn/ (W)

[sdeepaknarayanan]

Thanks for the response. :)