search

WillFlame14 / hanabi-bot

A bot that plays on the hanab.live interface.
GNU General Public License v3.0
14 stars 9 forks source link

Bot Evaluation #320

Closed tindiz closed 2 weeks ago

tindiz commented 2 weeks ago

Very interesting and cool project!

I was wondering if you have done evaluation for self play and/or cross play? Thought it would be interesting to see scores over n games for different conventions in self/cross play.

WillFlame14 commented 2 weeks ago

Thanks for the kind words!

There's a self-play command (documented in the README) that that can be customized for # of players and conventions. The performance fluctutates quite a bit since there are still a lot of bugs, but the bot is currently best at playing with H-Group conventions at level 7 or 11 with 3-4 players, usually scoring an average of 23-24 pts. It's noticeably worse at other player counts due to lack of tools (2p) and poor navigation (5p, 6p) but can still get > 20 pts on average. The Playful Sieve convention set is designed for only 2p, but I haven't worked on it as much and it currently performs slightly worse than H-Group.

Since this is a rule-based bot with no learning, I can't imagine cross-play going well. I have a basic mechanism in place for letting others make "mistakes", but the bot pretty much assumes everyone is following the same conventions it is.

tindiz commented 2 weeks ago

That's cool! Thanks for that!