agiresearch / WarAgent

WarAgent: LLM-based Multi-Agent Simulation of World Wars
Apache License 2.0
192 stars 29 forks source link

gt_edges and simulated_edges in evaluation_metric #6

Closed Gaotianhong closed 9 months ago

Gaotianhong commented 9 months ago

Hello, Thank you for great work!

I have question about how to get 'gt_edges' and 'simulated_edges' in evaluation_metric.py. Could you please give some tutorials?

Wenyueh commented 9 months ago

Hello,

Thank you for your question. The gt_edges refer to "ground_trueth edges", which are created based on real historical documentation.

For example in WWI, countries = [Britain, France, Germany, Austria, Russia, Serbia, USA, Ottoman] gt_edges_for_alliance = [(1,2), (3,4), (5,6), (2,5), (3,8)] gt_edges_for_war = [(4,6), (5,4), (3,5), (3,2), (1,3)] gt_edges_for_mobilization = [6,4,5,3,2,1,8]

The simulated edges are a length n list of lists, where n is the number of simulations you run under the same setting; each list represents your simulation result (on either alliance, war declaration, or mobilization).

Gaotianhong commented 9 months ago

Hello,

Thank you for your question. The gt_edges refer to "ground_trueth edges", which are created based on real historical documentation.

For example in WWI, countries = [Britain, France, Germany, Austria, Russia, Serbia, USA, Ottoman] gt_edges_for_alliance = [(1,2), (3,4), (5,6), (2,5), (3,8)] gt_edges_for_war = [(4,6), (5,4), (3,5), (3,2), (1,3)] gt_edges_for_mobilization = [6,4,5,3,2,1,8]

The simulated edges are a length n list of lists, where n is the number of simulations you run under the same setting; each list represents your simulation result (on either alliance, war declaration, or mobilization).

I get it, thank you for your reply.