sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.37k stars 466 forks source link

Solvers for constant sum games #18536

Closed 8ac3f0f3-bd13-47e7-baee-523ad1646342 closed 6 years ago

8ac3f0f3-bd13-47e7-baee-523ad1646342 commented 9 years ago

Constant-sum games are known to be solvable in polynomial time by using a linear program. This patch includes a solver which constructs and solves the LPs using the LP solvers within Sage (see http://doc.sagemath.org/html/en/reference/numerical/sage/numerical/mip.html). It also makes use of the solver within gambit for such games.

Finally, an additional function was included which helps to convert games from the representation in sage to gambits representation (_gambit_)

CC: @drvinceknight @dimpase @kcrisman @nathanncohen

Component: game theory

Keywords: Gambit, Zero-sum game Constant Sum Game, Normal Form Games

Author: Tobenna P. Igwe

Branch/Commit: 156ea0f

Reviewer: Karl-Dieter Crisman, Travis Scrimshaw, Dima Pasechnik, David Coudert

Issue created by migration from https://trac.sagemath.org/ticket/18536

kcrisman commented 9 years ago
comment:3

Various comments:

+        INPUT:
+
+        - ``solver`` - the solver to be used to solve the LP.
+
+          * ``'gambit'`` - This uses the solver included within the gambit library to
+            create and solve the LP.

same with other solvers.

+        if not self.is_constant_sum():
+            raise ValueError("Input game needs to be a two player constant sum game")

since presumably this is already tested for end users in _solve_LP, or do you think this sort of double-checking is needed? (In which case you might want to test both of those branches.)

+    def _as_gambit_game(self, as_integer=False):
+        r"""
+        Creates a Gambit game from a ``NormalFormGame`` object
            sage: c._solve_LCP(maximization=True) # optional - gambit
            [[(0.0, 1.0), (0.0, 1.0)]]

presumably still passes but what if one changed that to False?

+            if self.is_constant_sum():
+                algorithm = "lp-glpk"

        if algorithm.startswith('lp-'):
            return self._solve_LP(solver=algorithm[3:])

        if algorithm == "enumeration":
            return self._solve_enumeration(maximization)

and then else: blow up with a useful message

Finally, I haven't yet actually pulled anything nor verified that the MILP in question will solve constant-sum games, so that would remain as well to review this. Should be fun to get in, though, great to see stuff coming to Trac so early from the GSOC!!!

kcrisman commented 9 years ago

Reviewer: Karl-Dieter Crisman

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Changed commit from 1950054 to d957179

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

29d4a7dAdded tests for PPL and Coin-OR solvers
6138791Raise error if gambit isn't installed
0029b3fImproved documentation and doctests for _as_gambit_game
82da1bfRaise error upon wrong solver being passed
d957179Update check for constant sum 'is_constant_sum'
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Changed commit from d957179 to bb4eca8

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

bb4eca8Updated doctests for '_solve_LP'
8ac3f0f3-bd13-47e7-baee-523ad1646342 commented 9 years ago
comment:7

Thanks for your comments. I've implemented most of your comments, and noted a few extra things below which would be done upon feedback. If I've missed anything please let me know.

Replying to @kcrisman:

Various comments:

  • We have other LP solvers, might as well test that they actually work (with optional doctests of course). Surely Nathann will have interest in yet another use of them :)

Done some doctests for PPL and Coin-OR solvers. Tests would be included for CVXOPT once #18572 is done. ||29d4a7d||Added tests for PPL and Coin-OR solvers||

  • In _solve_gambit_LP do you need
+        if not self.is_constant_sum():
+            raise ValueError("Input game needs to be a two player constant sum game")

since presumably this is already tested for end users in _solve_LP, or do you think this sort of double-checking is needed? (In which case you might want to test both of those branches.)

I included it again just in case if the _solve_gambit_LP function was called externally without going through _solve_LP. If there is no need to retest just for this reason, then I'm happy to take it out.

  • What sort of error is raised if gambit isn't available for these LP things? Does it tell you to use gambit or does it say None has no such attribute or something?

Just added a ValueError to be raised which is quite similar to what you get by trying to solve the game with algorithm='LCP' option. ||6138791||Raise error if gambit isn't installed||

  • return c.numpy().max() == c.numpy().min() - is there no way to do this without using/importing numpy? It would be nice to not have to use it - or is it slower to use Sage proper?

Currently, this compares all entries of the matrix and makes sure it is within sys.float_info.epsilon of the first element. ||d957179||Update check for constant sum 'is_constant_sum'||

  • I don't mind in principle using the gambit conversion, obviously that is better when factored out, but then what happens to maximization in that case? Like
            sage: c._solve_LCP(maximization=True) # optional - gambit
            [[(0.0, 1.0), (0.0, 1.0)]]

presumably still passes but what if one changed that to False?

Currently, we are considering moving the maximization option into the constructor of the class.

  • Is this going to be more efficient in the constant-sum case even if lrs is installed? I just don't know the answer to relative efficiency here; presumably LP isn't always faster, even if often, but I don't know anything about lrs (point-counting?) either.
+            if self.is_constant_sum():
+                algorithm = "lp-glpk"

The LP solvers would probably be faster primarily because lrs enumerates all possible extreme Nash equilibria in a game, whereas the LP method simply finds one Nash equilibrium in the constant-sum game.

  • At this point there are so many options maybe one should also check for an invalid (read: mistyped) algorithm.

        if algorithm.startswith('lp-'):
            return self._solve_LP(solver=algorithm[3:])

        if algorithm == "enumeration":
            return self._solve_enumeration(maximization)

and then else: blow up with a useful message

||6138791||Raise error if gambit isn't installed||

There are two possible ValueError's that could be raised. The first comes from NormalFormGame upon entering a solver of the wrong format, and the second is from MixedIntegerLinearProgram if the LP solver is invalid.

kcrisman commented 9 years ago
comment:8

Thanks for your comments. I've implemented most of your comments, and noted a few extra things below which would be done upon feedback. If I've missed anything please let me know.

Great, very quick work.

I included it again just in case if the _solve_gambit_LP function was called externally without going through _solve_LP. If there is no need to retest just for this reason, then I'm happy to take it out.

Well, usually such underscore methods aren't "publicly" available. Vince, what do you think?

  • return c.numpy().max() == c.numpy().min() - is there no way to do this without using/importing numpy? It would be nice to not have to use it - or is it slower to use Sage proper?

Currently, this compares all entries of the matrix and makes sure it is within sys.float_info.epsilon of the first element. ||d957179||Update check for constant sum 'is_constant_sum'||

But which one is in principle faster or better? I don't mind using numpy as long as importing it doesn't cause problems in speed, if it's better (which perhaps it is).

Currently, we are considering moving the maximization option into the constructor of the class.

Fine, but this is currently breaking functionality. So either you have to do that here, or make that change a prereq to this ticket, or something else. My recommendation is to just leave it here for now and then deal with the class constructor bit in a separate ticket (to make things as orthogonal as possible).

The LP solvers would probably be faster primarily because lrs enumerates all possible extreme Nash equilibria in a game, whereas the LP method simply finds one Nash equilibrium in the constant-sum game.

Ah.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Changed commit from bb4eca8 to 60efdc7

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

60efdc7Fixed '_as_gambit_game' to support 'maximization' parameter
8ac3f0f3-bd13-47e7-baee-523ad1646342 commented 9 years ago
comment:10

I included it again just in case if the _solve_gambit_LP function was called externally without going through _solve_LP. If there is no need to retest just for this reason, then I'm happy to take it out.

Well, usually such underscore methods aren't "publicly" available. Vince, what do you think?

Actually, gambit performs it's own check as well, which makes three checks in total. So I've removed the one within _solve_gambit_LP, and added a doctest for the gambit error.

  • return c.numpy().max() == c.numpy().min() - is there no way to do this without using/importing numpy? It would be nice to not have to use it - or is it slower to use Sage proper?

Currently, this compares all entries of the matrix and makes sure it is within sys.float_info.epsilon of the first element. ||d957179||Update check for constant sum 'is_constant_sum'||

But which one is in principle faster or better? I don't mind using numpy as long as importing it doesn't cause problems in speed, if it's better (which perhaps it is).

The current implementation is faster than the numpy implementation. I ran some bench marks using timeit and got 177 µs per loop for the current implementation vs 992 ms per loop for the numpy implementation using a matrix of size 1000x1000.

Currently, we are considering moving the maximization option into the constructor of the class.

Fine, but this is currently breaking functionality. So either you have to do that here, or make that change a prereq to this ticket, or something else. My recommendation is to just leave it here for now and then deal with the class constructor bit in a separate ticket (to make things as orthogonal as possible).

Integrated the maximization parameter into _as_gambit_game.

kcrisman commented 9 years ago
comment:12

Great. Here is what I think still needs to happen.

ValueError: The Gambit implementation of LCP only
                                     allows for integer valued payoffs.
                                     Please scale your payoff matrices.
+        INPUT:
+        - ``as_integer`` - Boolean value which states whether the gambit representation should have
+                           the payoffs represented as integers or decimals.

(there should be a blank line between INPUT: and the rest, I think) even though it is an underscore method and won't appear in documentation.

Which is all not too hard, we're almost there.

What is the next project on the GSOC timeline?

kcrisman commented 9 years ago
comment:13

I cannot figure out how to get the changes in the doc built, it's maddening. I'll have to assume it's okay and I didn't miss anything.

The following could be improved, it really looks pretty meager. Not necessarily for this ticket, as definitely pre-existing - Vince, I'll blame you for this :)

class NormalFormGame(SageObject, MutableMapping):
    r"""
    An object representing a Normal Form Game. Primarily used to compute the
    Nash Equilibria.

    INPUT:

    - ``generator`` - Can be a list of 2 matrices, a single matrix or left
                      blank.

    """

It should at least tell how to get to more documentation from there.

The following

+        p = MixedIntegerLinearProgram(maximization=False, solver=solver)
+        y = p.new_variable(nonnegative=True)
+        v = p.new_variable(nonnegative=False)
+        p.add_constraint(self.payoff_matrices()[0] * y - v[0] <= 0)
+        p.add_constraint(matrix([[1] * strategy_sizes[0]]) * y == 1)
+        p.set_objective(v[0])

looks good though I would prefer to use maximization and swap that first constraint or whatever is appropriate, but presumably this is the industry standard to minimize.

So... is the reason for doing this only for constant-sum games because it's only known to be in P for them? Is it potentially still faster even for non-constant-sum games?

Doctests pass.

I think it's worth pointing out somewhere in the documentation that the LP approach will give one NE but not all of them, should there be more than one. (Constant-sum games can have more than one NE, right? I guess the trivial game is an example.)

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Changed commit from 60efdc7 to a24c7dd

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

0e300aeFixed indentation and removed incorrect error
313f42cUpdated tests for cbc and PPL
a24c7ddIncluded tests for constant-sum non-zero sum game and included maximization in the LP solver
8ac3f0f3-bd13-47e7-baee-523ad1646342 commented 9 years ago
comment:16
  • You should add an example testing the maximization=False

There's currently one at the start of obtain_nash, showing that it's possible for the equilibrium found could be different. Do you want me to do something similar for all the methods, or would the one be enough?

  • This error looks messed up because of the extra spaces - check for others like this:
ValueError: The Gambit implementation of LCP only
                                     allows for integer valued payoffs.
                                     Please scale your payoff matrices.

Actually, I was supposed to remove this error earlier on as it doesn't hold. I removed the documentation but forgot to remove the actual error.

  • You should decide whether you want # optional - Coin or # optional - cbc
  • Is PPL or CVXOPT really an optional thing? I think they are standard Sage packages...

I've removed #optional - PPL.

  • I (or another reviewer) need to check that the MILP is the right one
  • I (or another reviewer) need to check that documentation builds right and looks good
  • I (or another reviewer) need to check that the tests work right - guess I had better start installing some packages :)
  • I (or another reviewer) need to do some spot checks of everything

Which is all not too hard, we're almost there.

What is the next project on the GSOC timeline?

Coming up next would be the Lemke-Howson algorithm for solving 2-player games.

8ac3f0f3-bd13-47e7-baee-523ad1646342 commented 9 years ago
comment:17

The following could be improved, it really looks pretty meager. Not necessarily for this ticket, as definitely pre-existing - Vince, I'll blame you for this :)

class NormalFormGame(SageObject, MutableMapping):
    r"""
    An object representing a Normal Form Game. Primarily used to compute the
    Nash Equilibria.

    INPUT:

    - ``generator`` - Can be a list of 2 matrices, a single matrix or left
                      blank.

    """

It should at least tell how to get to more documentation from there.

OK. I was planning on opening a ticket, towards the end of GSOC, which would address the documentation as a whole. In the mean time, changes to the documentation would be based on changes made to the code.

The following

+        p = MixedIntegerLinearProgram(maximization=False, solver=solver)
+        y = p.new_variable(nonnegative=True)
+        v = p.new_variable(nonnegative=False)
+        p.add_constraint(self.payoff_matrices()[0] * y - v[0] <= 0)
+        p.add_constraint(matrix([[1] * strategy_sizes[0]]) * y == 1)
+        p.set_objective(v[0])

looks good though I would prefer to use maximization and swap that first constraint or whatever is appropriate, but presumably this is the industry standard to minimize.

So... is the reason for doing this only for constant-sum games because it's only known to be in P for them?

Yeah.

Is it potentially still faster even for non-constant-sum games?

Not sure if you are asking about using the LP for non-constant-sum games or the general classification of finding a Nash in a general game. In case if it was a bit of both:

That's Good.

I think it's worth pointing out somewhere in the documentation that the LP approach will give one NE but not all of them, should there be more than one. (Constant-sum games can have more than one NE, right? I guess the trivial game is an example.)

OK.

kcrisman commented 9 years ago
comment:18

OK. I was planning on opening a ticket, towards the end of GSOC, which would address the documentation as a whole.

Sounds fair.

Not sure if you are asking about using the LP for non-constant-sum games or the general classification of finding a Nash in a general game. In case if it was a bit of both:

  • LP's aren't guaranteed to find a Nash equilibrium in a two player non-constant-sum game.

Very interesting!

drvinceknight commented 9 years ago
comment:19

Hi both,

Apologies for my silence (on organisation committee of a conference that has been running this week so have been pretty busy). Apologies also if as a result my comments below don't make full sense (in case I missed something).

Replying to @kcrisman:

I cannot figure out how to get the changes in the doc built, it's maddening. I'll have to assume it's okay and I didn't miss anything.

The following could be improved, it really looks pretty meager. Not necessarily for this ticket, as definitely pre-existing - Vince, I'll blame you for this :)

class NormalFormGame(SageObject, MutableMapping):
    r"""
    An object representing a Normal Form Game. Primarily used to compute the
    Nash Equilibria.

    INPUT:

    - ``generator`` - Can be a list of 2 matrices, a single matrix or left
                      blank.

    """

It should at least tell how to get to more documentation from there.

This is certainly due to me, it happened as a result of moving the documentation that was there to the front matter, I think I perhaps did not fully understand :) Perhaps a ticket could be opened about the documentation (as Tobenna suggested towards the end of gsoc) that fully describes what we want the docs to look like :)

Replying to @ptigwe:

I think it's worth pointing out somewhere in the documentation that the LP approach will give one NE but not all of them, should there be more than one. (Constant-sum games can have more than one NE, right? I guess the trivial game is an example.)

OK.

I think it could be worth doctesting also? (So showing that all equilibria are find by some algorithms and not by LP etc...)

Great work on this Tobenna, really looking solid and thanks to Karl for reviewing (as always this is greatly appreciated).

kcrisman commented 9 years ago
comment:20

This is certainly due to me, it happened as a result of moving the documentation that was there to the front matter, I think I perhaps did not fully understand :) Perhaps a ticket could be opened about the documentation (as Tobenna suggested towards the end of gsoc) that fully describes what we want the docs to look like :)

Actually, I'll open it now and set it to sage-pending so we don't forget. #18609.

I think it's worth pointing out somewhere in the documentation that the LP approach will give one NE but not all of them, should there be more than one. (Constant-sum games can have more than one NE, right? I guess the trivial game is an example.)

OK.

I think it could be worth doctesting also? (So showing that all equilibria are find by some algorithms and not by LP etc...)

Okay, so we're agreed on this.

Another small point - no maximization here.

        if algorithm.startswith('lp-'):
            return self._solve_LP(solver=algorithm[3:])

Of course, at some point this is pedantic if you are planning on removing it - but actually, given that it pre-existed this ticket, I guess you'll have to deprecate that keyword.

You should add an example testing the maximization=False

There's currently one at the start of obtain_nash, showing that it's possible for the equilibrium found could be different. Do you want me to do something similar for all the methods, or would the one be enough?

What I meant was one for the new LP solver functionality.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

2f44485Tests for single / multiple Nash equilibria
fb4461cFixed minor error with LP solver
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Changed commit from a24c7dd to fb4461c

8ac3f0f3-bd13-47e7-baee-523ad1646342 commented 9 years ago
comment:22

Replying to @kcrisman:

Another small point - no maximization here.

        if algorithm.startswith('lp-'):
            return self._solve_LP(solver=algorithm[3:])

Of course, at some point this is pedantic if you are planning on removing it - but actually, given that it pre-existed this ticket, I guess you'll have to deprecate that keyword.

You should add an example testing the maximization=False

There's currently one at the start of obtain_nash, showing that it's possible for the equilibrium found could be different. Do you want me to do something similar for all the methods, or would the one be enough?

What I meant was one for the new LP solver functionality.

Dima, Vince and I have thought and discussed about this issue for a while, and have decided to remove maximization completely. So the next few patches should be addressing this.

kcrisman commented 9 years ago
comment:23

Okay. Probably best is to have that be a dependency of this ticket... And if you end up not deprecating this and just remove it (which could conceivably be reasonable in the particular context) be sure to have plenty of good arguments backing up that decision "for posterity's sake".

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

e4107dcUpdated tests for normal form games
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Changed commit from fb4461c to e4107dc

8ac3f0f3-bd13-47e7-baee-523ad1646342 commented 9 years ago
comment:25

Besides the deprecation (which would be done in a separate ticket), I believe that's everything for this one. If I've missed anything please let me know.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

c225b92Remove maximization from LP functions as it is going to be deprecated
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Changed commit from e4107dc to c225b92

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Changed commit from c225b92 to 93229b7

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

93229b7Revert "Remove maximization from LP functions as it is going to be deprecated"
tscrim commented 9 years ago
comment:30

Instead of calling the method _as_gambit_game, to be consistent with other parts of Sage, I would call it _gambit_ (like how the _gap_ method returns a representation of the object in GAP). Also the doc for INPUT: for that method is misaligned.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Changed commit from 93229b7 to ddd6f7e

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

ddd6f7eRenamed '_as_gambit_game' to '_gambit_' and fixed 'INPUT' indentation issues
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

6e2aae5Merge branch 'develop' into gt_extension
92345ccModified the '_gambit_' function to support n-player games
2c6aee7Update doctests
06d6b4cUpdated doctests of `catalog` to use `enumeration`
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Changed commit from ddd6f7e to 06d6b4c

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Changed commit from 06d6b4c to 2c4d951

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

2c4d951Updated front matter
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Changed commit from 2c4d951 to e538b14

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

e538b14Added name to AUTHORS
1adc0eef-8957-46d9-975b-2dd71dfbd9ba commented 8 years ago
comment:35

More doctests need to be marked as optional.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 8 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

51912a5Updated optional flags
90f7051Merge branch 'develop' into gt_extension
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 8 years ago

Changed commit from e538b14 to 90f7051

koffie commented 7 years ago
comment:38

Needs to be rebased on sage8.1.beta3

tscrim commented 6 years ago
comment:39

Rebased version.


New commits:

9b4a9cbRebased on 8.1.beta9.
tscrim commented 6 years ago

Changed branch from u/ptigwe/gt_extension to public/game_theory/solves_constant_sum_games-18536

tscrim commented 6 years ago

Changed reviewer from Karl-Dieter Crisman to Karl-Dieter Crisman, Travis Scrimshaw

tscrim commented 6 years ago

Changed commit from 90f7051 to 9b4a9cb

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 6 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

9e009fbSome small documentation updates.
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 6 years ago

Changed commit from 9b4a9cb to 9e009fb