causalincentives / pycid

Library for graphical models of decision making, based on pgmpy and networkx
Apache License 2.0
96 stars 13 forks source link

Solubility and control ordering #65

Closed sbenthall closed 1 year ago

sbenthall commented 2 years ago

Is your feature request related to a problem? Please describe. For a given CID, it's of interest to know whether or not it is soluble, as defined by Nilsson and Lauritzen (2000). It is also useful to know what in order of decision nodes a soluble model can be solved via backwards induction.

Describe the solution you'd like Maybe a method on the CID class that tests for solubility, returning an ordering of decision nodes if it's soluble and None otherwise?

Describe alternatives you've considered I wonder if this capability is already in the library.

Additional context I've been wondering if solubility has been defined for MACIDs. I believe a MACID should be soluble if for each agent, its decision nodes are soluble with respect to its own utility nodes, while presuming all other control nodes are turned into chance nodes by a fixed decision strategy. While that does not afford backwards induction to solve the entire model, it would allow the backwards induction of each agent given the strategy of others, which could then be iterated until equilibrium.

Or, a strong version of multi-agent solubility would be the original condition, but considering all utility and decision nodes regardless of the agent that owns them. This would identify a much smaller class of CIDs as soluble, but it might be possible to solve these for all agents with a single backwards induction pass.

RyanCarey commented 2 years ago

Yes, we do have one that treats single and multi agent cases, just it's named "sufficient recall https://github.com/causalincentives/pycid/blob/master/pycid/core/macid_base.py#L257". We should definitely mention in the docstring that sufficient recall is also known as solubility.

I believe a MACID should be soluble if for each agent, its decision nodes are soluble with respect to its own utility nodes, while presuming all other control nodes are turned into chance nodes by a fixed decision strategy. Yes, absolutely, and this is how it's defined in the literature, for instance definition 5.2 of "Ignorable Information in Multi-Agent Scenarios" by Milch & Koller.

The stronger definition sounds plausible.

On Fri, 20 May 2022 at 23:06, Sebastian Benthall @.***> wrote:

Is your feature request related to a problem? Please describe. For a given CID, it's of interest to know whether or not it is soluble, as defined by Nilsson and Lauritzen (2000). It is also useful to know what in order of decision nodes a soluble model can be solved via backwards induction.

Describe the solution you'd like Maybe a method on the CID class that tests for solubility, returning an ordering of decision nodes if it's soluble and None otherwise?

Describe alternatives you've considered I wonder if this capability is already in the library.

Additional context I've been wondering if solubility has been defined for MACIDs. I believe a MACID should be soluble if for each agent, its decision nodes are soluble with respect to its own utility nodes, while presuming all other control nodes are turned into chance nodes by a fixed decision strategy. While that does not afford backwards induction to solve the entire model, it would allow the backwards induction of each agent given the strategy of others, which could then be iterated until equilibrium.

Or, a strong version of multi-agent solubility would be the original condition, but considering all utility and decision nodes regardless of the agent that owns them. This would identify a much smaller class of CIDs as soluble, but it might be possible to solve these for all agents with a single backwards induction pass.

— Reply to this email directly, view it on GitHub https://github.com/causalincentives/pycid/issues/65, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB7ULO6JAEMJTUZFEX5MZH3VLAEGNANCNFSM5WQYPXWA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- Ryan Carey FHI Research Fellow & Statistics DPhil Student, Oxford University

Jamesfox1 commented 2 years ago

Yep, agreed with @RyanCarey. I will edit the docstring. We determine sufficient recall/agent solubility using the "relevance graph". If the relevance graph restricted to just that agent's decision nodes is acyclic, then it has sufficient recall. Your latter condition would check if the relevance graph including all decision nodes was acyclic. A method can easily be constructed for this, but it's not clear why it's needed because sufficient recall is alone sufficient for the existence of a pure subgame perfect equilibrium.

sbenthall commented 2 years ago

Aha, thank you. I didn't know solubility was the same thing as sufficient recall, or that it had already been developed in the multi-agent case by Milch & Koller. I'll read up. Yes, if the comments had mentioned this, I would have been able to find it!

Jamesfox1 commented 1 year ago

I've updated the doc string now