Replicating Benchmark Results between Piosolver and TexasSolver

[def-mycroft]

I'll preface this by communicating that I have no reason to doubt the accuracy of the solver built here and that I don't doubt it, I just want to understand the possible causes of some differences between piosolver and TexasSolver (also, I get that different algos can produce equally unexploitable and non-identical strategies).

I want to use TexasSolver to generate a large number of solves, but need to make sure I fully understand what is going on before making that investment. Thanks in advance for any input, and, again, I really appreciate the work that has been done here and the sharing of it.

I wanted to try to replicate the benchmark test mentioned in the README. I used the config files that are referred to in the README in that section. I changed the TexasSolver config to dump 3 streets instead of two, and I removed the suppression of 3bets from the piosolver config (as it doesn't appear that this is applied in Texas Solver). Here are the params I used for TexasSolver and here are the params used for piosolver.

For OOP's first action on the flop (not pictured), I notice a few differences, but all of these were differences in frequencies and would be unlikely to matter very much, e.g. checking QQ @ a rate of 46% vs a rate of 72% is kind of a large difference, but in practice it is difficult to implement that kind of fine-grained difference and would seem to be impossible for an opponent to figure that out and exploit.

On other nodes I've observed larger differences; e.g. in the image below, piosolver has IP raising OOP's bet ~7% of the time, and even pure betting some combos of 22; while TexasSolver is generally close, but doesn't have IP raising at all.

I did try solving with the two different algos available in piosolver nowdays, i.e. PIO_CFR and the original pio algo; both produced very similar results.

What might account for the observation that piosolver has IP raising 7% in this spot and TexasSolver has IP raising 0%?

[bupticybee]

Oh, you can call it kind of a bug, TexasSolver's game tree construction algorithm is very different from piosolver's. I think that's why they produce very different results.

I will find a time to modify the game tree generating algorithm.

Or if you are familar with c++ you can try modify them yourself:

https://github.com/bupticybee/TexasSolver/blob/master/src/GameTree.cpp#L31

Cause I'm kind of busy rescently. Anyway thanks very much for you feedbacks. I will leave the issue open for the time and I think we will find some solution in the near future.

[xuzy1975]

I know what happened, config item "set_raise_limit 5" should be added.

[bupticybee]

I know what happened, config item "set_raise_limit 5" should be added.

Thank you for clearification @def-mycroft please check whether this solve your problem.

[zachhilbert]

@bupticybee you mention modifying the game tree construction algorithm. How would the optimal design differ from the current implementation? Basically, what needs to change in order to accomplish this, and is it necessary? I have read through most of the code to understand how everything works, but I'm not familiar enough to validate whether the algorithms for calculating frequencies, bet sizes, etc are accurate.

[bupticybee]

@bupticybee you mention modifying the game tree construction algorithm. How would the optimal design differ from the current implementation? Basically, what needs to change in order to accomplish this, and is it necessary? I have read through most of the code to understand how everything works, but I'm not familiar enough to validate whether the algorithms for calculating frequencies, bet sizes, etc are accurate.

The job is to modify the tree building algorithm to generate the same tree as piosolver does. My suggestion is first to find the different tree structure compair to piosolver and then try mess with the code the make the tree structure the same with piosolver. Not sure if I explain myself clear.