vocab_size parameter

[mitjanikolaus]

If I understand correctly, the vocab_size command line parameter defines the number of symbols that the agents use for communication, including the EOS symbol (0). Is this intentional? Say, we want agents to communicate about 100 distinct values (and multiple attributes), intuitively we'd need to set the vocab size to minimally 101 to have allow the agents to communicate successfully? (given max_len = # attributes).

[robertodessi]

In the case of messages of length greater than one (meaning you are using some RNN-based message generator in EGG) that should be without EOS. We're appending EOS at the end of the generated message. https://github.com/facebookresearch/EGG/blob/main/egg/core/reinforce_wrappers.py#L343-L345 A message is generated by unrolling for max_len number of steps.

Note that the agent might send an EOS before the last timestep. In that case when we are computing msg_len we're ignoring anything after that EOS.

With gumbel-based optimization things are a bit trickier but we do the same things with a slightly different logic.

Hope this is clear

[mitjanikolaus]

Thanks for the very fast response! :)

Yes, I'm talking about some RNN-based message generator with length greater 1. My question is not regarding the message length, but the vocab_size. If one of the tokens (0) is used as EOS token, the effective vocab size is actually only vocab_size-1, right? So if if set vocab_size to 100, and given that the agent wants to use the whole message length and not send the EOS token before reaching max_len, it can only send one of 99 different symbols every timestep. Do I understand that correctly?

[robertodessi]

Oh yes sorry, I went straight to max_len when you were talking about the vocab_size.

Yes that's correct. However, I think it depends whether you consider EOS as part of the vocab_size. It does carry a meaning that could/should be understood from the receiver in a communication game, hence I think it makes sense to count it as part of the vocabulary. If you don't consider it so then yes, there are only 99 possible symbols with a vocab_size of 99.

[mitjanikolaus]

Alright, yes, I it's definitely part of the vocabulary, but just kind of a special token because you can't send other tokens after it. Maybe it would be good to make this more transparent, by expanding the description of the command line argument?

E.g., like this: https://github.com/facebookresearch/EGG/commit/7520036b0bd6f900d9b38411f5f96367ccefa3aa

[robertodessi]

Sure, feel free to open a PR with this

[robertodessi]

fixed in #249