Open jbirkesteen opened 1 year ago
Also for HumanPlayer, it is not possible to move the game while choosing a column and it chrashed my program when I tried to choose a column.
Could the issue be rooted in Env.step()
? In this method, we check if a move is legal and then place it no matter what. Just reading it, it seems like it would give an error, but I haven't tested it yet.
The new implementation failed when the DirectPolicyAgent chose an illegal column in an environment without illegals allowed. All entries in
probs
were 0 after the list comprehension. The method was out-commented and replaced with the old approach (random choice) with bbe6c62.We didn't really figure out what went wrong.
Another suggestion than purely random: We could keep drawing from probs (for instance, a maximum of 5 times) with categorical until it (by chance) draws a legal column. This of course only makes sense before the algorithm converges to having very high probabilities.