search

google-deepmind / concordia

A library for generative social simulation
Apache License 2.0
633 stars 148 forks source link

'Too many multiple choice attempts.' error #56

Closed anatoleg closed 5 months ago

anatoleg commented 5 months ago

The sample_choice method in gpt_model.py throws the 'Too many multiple choice attempts.' error. Here is the prompt:

Status of players: Bob is not mentioned to be anywhere or doing anything at the current time. His latest known location and activity will be at the Sundrop Saloon waiting out a snow storm on 02 Oct 2024 18:00:00. Alice is not mentioned to be in any specific location or doing anything at the current time. Charlie is not mentioned to be anywhere or doing anything at the current time. The latest information about him is that he will be at the Sundrop Saloon on 02 Oct 2024, waiting out a snow storm. Dorothy is not mentioned in the latest event. Her latest location and activity are unknown.

Question: Does the following event directly affect anyone from this list? List: ['Bob', 'Alice', 'Charlie', 'Dorothy']. Event: At the Sundrop Saloon, Bob attempts to approach Alice to discuss the car accident and negotiate a payment plan for the damages, but is unable to find her. (a) Yes (b) No Answer: (a)

Question: Does the event affect Bob? (a) No (b) Yes Answer: ( Respond EXACTLY with one of the following strings: a b.

The sample_text method returns string 'b)' instead of 'b' and it does it 3 times triggering the error. The same prompt may also returns 'b) Yes' I am using GPT-4 Should I use sample[0] instead of sample? This seems like a hack. Any suggestions?

jzleibo commented 5 months ago

This was already fixed. It sounds like you have an old version. Try updating to the latest.

anatoleg commented 5 months ago

The fix was made after the latest package release 1.3.0.

jzleibo commented 5 months ago

Thanks, I'll try and update that as soon as I have a moment, it's on my list.

jzleibo commented 5 months ago

I just updated the release package to 1.4.0, so it now contains all the latest changes, including the GPT model wrapper fix. Thanks for nudging me to get this done :)