search

nkg114mc / c-gomoku-cli

Command Line Interface tool for Gomocup gomoku/renju engines
GNU General Public License v3.0
7 stars 3 forks source link

[Enhancement] Consider utilizing position chains when saving training samples #11

Open dhbloo opened 3 years ago

dhbloo commented 3 years ago

Currently c-gomoku-cli saves training samples by seperately saving each positions' full move sequence and result. This introduces a lot of redundency in the saved file, which is why LZ4 compression is introduced. However, this redundency can be reduced by considering chaining info in various positions in one continous game. If this format is in use, LZ4 compression is no longer needed.

For example, all those positions should be chained together:

...
f10g8i6i9f8h9g7f7
f10g8i6i9f8h9g7f7i10g9f9
f10g8i6i9f8h9g7f7i10g9f9h11e9d10
f10g8i6i9f8h9g7f7i10g9f9h11e9d10g11h12
f10g8i6i9f8h9g7f7i10g9f9h11e9d10g11h12h10k9
f10g8i6i9f8h9g7f7i10g9f9h11e9d10g11h12h10k9j9j10
f10g8i6i9f8h9g7f7i10g9f9h11e9d10g11h12h10k9j9j10i11
f10g8i6i9f8h9g7f7i10g9f9h11e9d10g11h12h10k9j9j10i11k11l12
f10g8i6i9f8h9g7f7i10g9f9h11e9d10g11h12h10k9j9j10i11k11l12k8k7
f10g8i6i9f8h9g7f7i10g9f9h11e9d10g11h12h10k9j9j10i11k11l12k8k7g10h6i5
...

Stockfish is also using this approach to save storage used for saving training samples.