notnil / chess

chess package for go
MIT License
517 stars 130 forks source link

Reading large PGN with files #47

Closed jmellian closed 4 years ago

jmellian commented 4 years ago

ISSUE #46 Added a splitfunction pgnSplit for a multi game pgn-file. Add Testfunction TestPGNScanner

This request has come up a few times so it is definitely something people are interested in. I think a streaming parser makes sense in this case. I also think the API will be more ergonomic than a limited reader. It could look something like:

f, _ := os.Open("many.pgn") scanner := chess.NewPGNScanner(f) for scanner.Scan() { game := scanner.Next() fmt.Println(game) } if err := scanner.Err(); err != nil { fmt.Fprintln(os.Stderr, "reading standard input:", err) }

ISSUE #32
I'd like to use this library to read really large PGN files. There's two suggestions I'd like to make to improve this. If you agree with my suggestions, then I'm happy to have a crack at implementing them: 1). Stop logging import messages, this slows things down.
2). Create a PGN reader class, similar to LimitedReader, that allows you to read up to a fixed number of games at a time so that you don't have to have all the games in memory at once. Func LimitedGamesFromPGN is implementing issue #32 point 2. Func LimitedGamesFromPGNReader gets rid of importing messages. Two Unittest are added for LimitedGamesFromPGNReader and GamesFromPGN. They are using a large pgn file which is added in assets: twic0920.pgn (2245 games from This week in Chess)