Open lwaekfjlk opened 1 month ago
class name including EvaluationForTwoAgents is not supported
run experimental_eval.py with sotopia==0.0.11
independently install sotopia
python experimental_eval.py \ --gin_file sotopia_conf/generation_utils_conf/generate.gin \ --gin_file sotopia_conf/server_conf/server.gin \ --gin_file sotopia_conf/run_async_server_in_batch.gin \ '--gin.ENV_IDS=["01H7VFHNV13MHN97GAH73E3KM8", "01H7VFHN5WVC5HKKVBHZBA553R", "01H7VFHNN7XTR99319DS8KZCQM", "01H7VFHN9W0WAFZCBT09PKJJNK", "01H7VFHPDZVVCDZR3AARA547CY", "01H7VFHPQQQY6H4DNC6NBQ8XTG", "01H7VFHPQQQY6H4DNC6NBQ8XTG", "01H7VFHN7WJK7VWVRZZTQ6DX9T", "01H7VFHN7A1ZX5KSMT2YN9RXC4", "01H7VFHPS5WJW2694R1MNC8JFY", "01H7VFHPS5WJW2694R1MNC8JFY", "01H7VFHNN7XTR99319DS8KZCQM", "01H7VFHQ11NAMZS4A2RDGDB01V", "01H7VFHQ11NAMZS4A2RDGDB01V", "01H7VFHPSWGDGEYRP63H2DJKV0", "01H7VFHPSWGDGEYRP63H2DJKV0", "01H7VFHNF4G18PC9JHGRC8A1R6", "01H7VFHNNYH3W0VRWVY178K2TK", "01H7VFHP8AN5643B0NR0NP00VE", "01H7VFHN7A1ZX5KSMT2YN9RXC4"]' \ '--gin.AGENT1_MODEL="gpt-3.5-turbo"' \ '--gin.AGENT2_MODEL="gpt-3.5-turbo"' \ '--gin.BATCH_SIZE=1' \ '--gin.PUSH_TO_DB=True' \ '--gin.TAG="gpt-3.5-turbo_vs_gpt-3.5-turbo-0927"'
No response
@lwaekfjlk is this fixed or still there? imo it's bc we removed the retrieved prompt there and we are still figuring out a way to fix this.
still there I believe
Description of the bug
class name including EvaluationForTwoAgents is not supported
Steps To Reproduce
run experimental_eval.py with sotopia==0.0.11
independently install sotopia
python experimental_eval.py \ --gin_file sotopia_conf/generation_utils_conf/generate.gin \ --gin_file sotopia_conf/server_conf/server.gin \ --gin_file sotopia_conf/run_async_server_in_batch.gin \ '--gin.ENV_IDS=["01H7VFHNV13MHN97GAH73E3KM8", "01H7VFHN5WVC5HKKVBHZBA553R", "01H7VFHNN7XTR99319DS8KZCQM", "01H7VFHN9W0WAFZCBT09PKJJNK", "01H7VFHPDZVVCDZR3AARA547CY", "01H7VFHPQQQY6H4DNC6NBQ8XTG", "01H7VFHPQQQY6H4DNC6NBQ8XTG", "01H7VFHN7WJK7VWVRZZTQ6DX9T", "01H7VFHN7A1ZX5KSMT2YN9RXC4", "01H7VFHPS5WJW2694R1MNC8JFY", "01H7VFHPS5WJW2694R1MNC8JFY", "01H7VFHNN7XTR99319DS8KZCQM", "01H7VFHQ11NAMZS4A2RDGDB01V", "01H7VFHQ11NAMZS4A2RDGDB01V", "01H7VFHPSWGDGEYRP63H2DJKV0", "01H7VFHPSWGDGEYRP63H2DJKV0", "01H7VFHNF4G18PC9JHGRC8A1R6", "01H7VFHNNYH3W0VRWVY178K2TK", "01H7VFHP8AN5643B0NR0NP00VE", "01H7VFHN7A1ZX5KSMT2YN9RXC4"]' \ '--gin.AGENT1_MODEL="gpt-3.5-turbo"' \ '--gin.AGENT2_MODEL="gpt-3.5-turbo"' \ '--gin.BATCH_SIZE=1' \ '--gin.PUSH_TO_DB=True' \ '--gin.TAG="gpt-3.5-turbo_vs_gpt-3.5-turbo-0927"'
Additional Information
No response