[bug] Nondeterminism

[AlekseyKorshuk]

I have spotted some lines where the generator is used without the seed. Some of them might not be used, but the rest a critical for result consistency. Here is the list:

• https://github.com/google-deepmind/concordia/blob/9f317f53248e4905e336be89780d991f39bd5171/examples/modular/environment/forbidden_fruit.py#L112
• https://github.com/google-deepmind/concordia/blob/9f317f53248e4905e336be89780d991f39bd5171/examples/modular/environment/forbidden_fruit.py#L897
• https://github.com/google-deepmind/concordia/blob/9f317f53248e4905e336be89780d991f39bd5171/concordia/components/agent/person_representation.py#L109
• https://github.com/google-deepmind/concordia/blob/9f317f53248e4905e336be89780d991f39bd5171/concordia/components/game_master/conversation.py#L188
• https://github.com/google-deepmind/concordia/blob/9f317f53248e4905e336be89780d991f39bd5171/concordia/contrib/components/game_master/daily_activities.py#L399
• https://github.com/google-deepmind/concordia/blob/9f317f53248e4905e336be89780d991f39bd5171/concordia/contrib/components/game_master/world_background_and_relevance.py#L122
• https://github.com/google-deepmind/concordia/blob/9f317f53248e4905e336be89780d991f39bd5171/concordia/contrib/components/game_master/world_background_and_relevance.py#L155
• https://github.com/google-deepmind/concordia/blob/9f317f53248e4905e336be89780d991f39bd5171/concordia/document/interactive_document.py#L274
• https://github.com/google-deepmind/concordia/blob/9f317f53248e4905e336be89780d991f39bd5171/concordia/thought_chains/thought_chains.py#L366
• https://github.com/google-deepmind/concordia/blob/9f317f53248e4905e336be89780d991f39bd5171/examples/modular/environment/modules/player_traits_and_styles.py#L191
• https://github.com/google-deepmind/concordia/blob/9f317f53248e4905e336be89780d991f39bd5171/examples/modular/environment/modules/player_traits_and_styles.py#L193
• https://github.com/google-deepmind/concordia/blob/9f317f53248e4905e336be89780d991f39bd5171/examples/modular/environment/modules/player_traits_and_styles.py#L206
• https://github.com/google-deepmind/concordia/blob/9f317f53248e4905e336be89780d991f39bd5171/examples/modular/environment/modules/pub_coordination_capetown.py#L200
• https://github.com/google-deepmind/concordia/blob/9f317f53248e4905e336be89780d991f39bd5171/examples/modular/environment/modules/pub_coordination_london.py#L283
• https://github.com/google-deepmind/concordia/blob/9f317f53248e4905e336be89780d991f39bd5171/examples/open_ended/open_ended.py#L178

[jzleibo]

Here's how we're thinking about this. We could make the lines you mention use the seed, and we should and will do that in the long run. However, this won't actually make the result deterministic. The problem is that these are all places that will be called after substantial use of the LLM API has already happened. But the LLM API could be delayed for network traffic reasons, rate limits, etc, any number of things. These would all end up advancing the seed. So just making the lines you mention take a seed parameter would not actually yield a deterministic result in practice.

What we have done to take as big a step as possible toward determinism is to make sure that all the random choices that happen in environment setup, before we use the LLM, are all seeded properly. This part of the variation really can be made deterministic, so we have done that.