softmax的近似之NCE详解 — carlos9310

[carlos9310]

https://carlos9310.github.io/2019/10/15/Approximating-the-Softmax/

深度学习中与分类相关的问题都会涉及到softmax的计算。当目标类别较少时，直接用标准的softmax公式进行计算没问题，当目标类别特别多时，则需采用估算近似的方法简化softmax中归一化的计算。

[18810217989]

写的真棒，终于看懂NCE了

[luoshao23]

赞！

[priyaexcelr]

Amazing Article ! I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up. Correlation vs Covariance Simple Linear Regression data science interview questions KNN Algorithm Logistic Regression explained

[Berwin77]

非常清晰

[flyto22c]

-博主好，我想问下公式 exp(h^T v_w^') 中的 h 是什么？是上下文单词 c 的 embedding 吗？还是单独定义的一个向量的训练参数？

-另外可以在NEC公式原理中讲讲 bias 部分应该在哪些位置加上去吗？博文开头逻辑回归中的bias部分也显示写出来可能理解更好呀！

[CaraJ7]

Hi! Thanks for the detailed introduction. Minor problem here. Maybe the 6th equation in the article, which is shown as $$P(y,w|c) = \frac{1}{1+k}P{train}(w|c)+\frac{k}{1+k}Q(w)$$ should be $$P(w|c) = \frac{1}{1+k}P{train}(w|c)+\frac{k}{1+k}Q(w)$$ There shouldn't be a $y$ in the left side of the equation.