search

official-stockfish / nnue-pytorch

Stockfish NNUE (Chess evaluation) trainer in Pytorch
GNU General Public License v3.0
320 stars 99 forks source link

Add blocked sparse input affine transform approach to docs. #249

Closed Sopel97 closed 1 year ago

Sopel97 commented 1 year ago

Based on https://github.com/official-stockfish/Stockfish/pull/4612

vondele commented 1 year ago

I have read through this, looks OK.

Independent from this PR, should we integrate the permutation code (as mentioned https://github.com/official-stockfish/Stockfish/pull/4620#issue-1757202863) in the trainer somehow ?

Sopel97 commented 1 year ago

Independent from this PR, should we integrate the permutation code (as mentioned https://github.com/official-stockfish/Stockfish/pull/4620#issue-1757202863) in the trainer somehow ?

We could maybe apply it during serialization if we have a good enough solution. Right now we should work on a separate tool. I made some initial scaffolding here https://github.com/Sopel97/nnue-pytorch/blob/ftperm/ftperm.py

vondele commented 1 year ago

I think the greedy permutation that is used in the SF PR mentioned is likely a near-optimal solution to do this.

Computing the permutation needs some 50M positions, which in principle we can do by just using that many from a binpack and doing inference. Right now this is some modified version of SF collecting that data, but that's not ideal.

vondele commented 1 year ago

anyway, docs seem complete. Ready for a merge?

Sopel97 commented 1 year ago

yes