jwijffels
commented
3 years ago Interesting. I think (don't remember today) I've taken some default settings from https://github.com/hiyijian/doc2vec/blob/master/test/TestTrain.cpp#L8 but maybe I looked as well to the gensim defaults at https://github.com/RaRe-Technologies/gensim/blob/develop/gensim/models/doc2vec.py#L165 but would certainly be interested in finding out links to other default settings.
Finally a solid doc2vec implementation in R. Many thanks! I have a relatively minor suggestion: I feel that the default parameter values might be underselling the power of this method. I know everyone can change the default settings, but in reality most users just want to "press play". When I look at most doc2vec applications in Python - the go to text analysis language for most - they go for more demanding settings. For example, the top2vec module uses roughly the following default parameter values (from https://github.com/ddangelov/Top2Vec/blob/master/top2vec/Top2Vec.py):
These values are surely not 100% scientific, but I think the authors have experimented quite a bit before arriving to them. I think they are a useful starting point.
The default values as you have them now make the process very fast but the resulting embeddings might often be quite poor. Negative subsampling, in particular, has been in some contexts associated with hurting the quality of the semantic space. I can also say that in my use case the default settings are not ideal, while the ones above yield pretty solid results within a reasonable time. Just a suggestion.