Definition of minCountLabel

[budiryan]

What is the definition of minCountLabel? Isn't this algorithm supposed to be unsupervised?

Thanks!

[kaushikacharya]

@budiryan As per my understanding, minCountLabel is the min word count threshold.

Have a look at its usage: https://github.com/epfml/sent2vec/blob/master/src/fasttext.cc#L473

    if (uniform(model.rng) > dict_->getPDiscard(line[i]) || dict_->getTokenCount(line[i]) < args_->minCountLabel)
      continue;

Here line represents the vector of words( to be specific its hash index).

Following code snippet shows how the word index is extracted: https://github.com/epfml/sent2vec/blob/master/src/dictionary.cc#L30

int32_t Dictionary::find(const std::string& w) const {
  int32_t h = hash(w) % MAX_VOCAB_SIZE;

getTokenCount is defined in https://github.com/epfml/sent2vec/blob/master/src/dictionary.cc#L133

int64_t Dictionary::getTokenCount(int32_t id) const {
    assert(id >= 0);
    assert(id < size_);
    return words_[id].count;
}

[martinjaggi]

yes, thanks @kaushikacharya

and yes the algorithm is unsupervised (contrastive or self-supervised learning).

can i close this?

[budiryan]

Thank you @kaushikacharya and @martinjaggi for replying!

I have further question:

Then how will it differ from minCount?

It seems that minCount is only used here: threshold(args_->minCount, args_->minCountLabel);

What is the purpose of that function? I am having trouble understanding it

[kaushikacharya]

@budiryan Seems you have raised a valid doubt.

sent2vec source code is developed over the existing C++ code of fastText. Hence it has several sections which are not used in sent2vec e.g. supervised training.

https://github.com/facebookresearch/fastText/issues/530#issuecomment-394816034 Edouard Grave has mentioned that minCountLabel is to be used for supervised mode only.

Based on this observation, my understanding is that minCountLabel should be replaced by minCount in the functions cbowCWNgrams and sent2vec

i.e. in the following lines https://github.com/epfml/sent2vec/blob/master/src/fasttext.cc#L439 if (uniformSub(model.rng) > dict_->getPDiscard(line[i]) || dict_->getTokenCount(line[i]) < args_->minCountLabel) {

https://github.com/epfml/sent2vec/blob/master/src/fasttext.cc#L473 if (uniform(model.rng) > dict_->getPDiscard(line[i]) || dict_->getTokenCount(line[i]) < args_->minCountLabel)

In threshold(args_->minCount, args_->minCountLabel) minCount acts as threshold for word token and minCountLabel acts as threshold for labels (supervised mode).

https://github.com/epfml/sent2vec/blob/master/src/dictionary.cc#L276 void Dictionary::threshold(int64_t t, int64_t tl) {