lorserker / ben

a game engine for bridge
GNU General Public License v3.0
41 stars 30 forks source link

Final double is missing #41

Closed ThorvaldAagaard closed 1 year ago

ThorvaldAagaard commented 1 year ago

During test of #38 I once missclicked in the bidding resulting in this sequence image

I know this sequence will not be present in the training dataset, so it is understandable that BEN will be in trouble.

Perhaps East expects West to double, but West is making the final decision and here it seems only Pass is a possible bid: image The simulated hands for West does not match the bidding very good, but the real problem seems to be that Double isn't evaluated

lorserker commented 1 year ago

yes, i have seen this bug in other situations too. the problem is that the model suggests pass so strongly that nothing else is evaluated. if double was evaluated, it very likely would be selected.

lorserker commented 1 year ago

i think it is a possible solution to add some heuristic to force consideration of double if the opponents are known to be much weaker or in other situations to consider more bids if it looks like we end up in a contract with few trumps.

ThorvaldAagaard commented 1 year ago

I think that is a good idea.

I just tested a known bug from GIB, where you as opponent bid 4N over an 2Club opening. It resulted in this: image

I am still a bit confused about the randomness of the bids, as when I reran this board North now bid 5D (And now East doubled)

Is there some random seed I can set, so I always get the same output on a given input?

lorserker commented 1 year ago

@ThorvaldAagaard the randomness comes from the sampling. it is possible to seed the random number generator, but unfortunately I didn't do it when I wrote the code.

ThorvaldAagaard commented 1 year ago

Adding

np.random.seed(42)

in the Driver Class init method in game.py seems to fix the problem.

image

I can see that there is a mix of use of random and np.random, so I also changed all use of random to np.random (Could also seed random in the Driver Class, but I think it is better just to use one random-function)

I will take a look at how find patterns, where a Double, Redouble or Pass should be considered when not suggested by the Neural network

ThorvaldAagaard commented 1 year ago

I have added a principle of lowering the threshold in the pass out situation, so BEN will now double 3N, and also find a double after the 2C-opening.

Interesting if we will see many more penalty doubles.