search

Cryolite / kanachan

A Japanese (Riichi) Mahjong AI Framework
285 stars 40 forks source link

What is the performance of the current model? #11

Open kaen249 opened 2 years ago

kaen249 commented 2 years ago

I couldn't find a place to ask, so I'm asking you here.

  1. I haven't trained yet, but I'm curious about the kanachan's performance.
  2. If I have finish training, How can I test the trained model?
Cryolite commented 2 years ago

In order to measure the objective performance of kanachan, a baseline mahjong AI model that satisfies the following conditions will be needed:

  1. it performs to a certain degree,
  2. the model is publicly available, and
  3. its performance on major online mahjong platforms such as Tenhou and Majsoul is roughly known.

I am currently planning to use akochan for performance measurement as a baseline satisfying the above conditions. However, since akochan is based on the mjai protocol but kanachan's mjai support has not been implemented yet, the objective performance of kanachan is still unknown. Please wait a little longer for reports on kanachan's objective performance.

Once I implement performance comparison with akochan, I will publish it in this repository so that you can use it for objective performance measurement.