niklasf / python-chess

A chess library for Python, with move generation and validation, PGN parsing and writing, Polyglot opening book reading, Gaviota tablebase probing, Syzygy tablebase probing, and UCI/XBoard engine communication
https://python-chess.readthedocs.io/en/latest/
GNU General Public License v3.0
2.44k stars 531 forks source link

Peformance #1

Closed nate1001 closed 11 years ago

nate1001 commented 11 years ago

Hi, I have been examining your chess library and I think it is quite good. I have not found any good, readable x88 chess validation libraries implemented for python. However, due to the way the classes are setup, the x88 speed gain is completely overshadowed by attribute access and object creation.

Testing the making of 10 moves (Move.from_uci not from SAN) took over 3 seconds on my somewhat slow netbook (AMD C60). Tying this in a gui interface causes notable lag on adding a move and validating a game of 30 moves would be an hourglass operation.

I see that you have heavily used custom objects with "private" attribute access. While this may lead to better OO design, in Python this comes at a performance cost. I made a simple test case http://pastebin.com/0x0xpVLv of atribute access vs. property access. At 100,000 iterations, the difference of adding 1 to an attribute via property vs attribute was .361 and .063 seconds respectively (a 5x speedup). Profiling the position.py code, in 10 moves the inner functions were being called around 20,00 times.

The same will hold true for object creation where the profiler shows the code in init's a significant amount of time.

I forked and rewrote the classes using implied privacy and held the underlaying data structure as the x88 number in Square and Move so it would not have to be calculated. I switched the x88 pieces list from holding Piece objects to 1 length pieces strings. I used copy.deepcopy to copy the Position to avoid parsing and recreating the fen string. I only created the squares one time in and held them as a class variable as I consider them immutable.

All this produced about a 10 times speed up with only trivial changes to the x88 code! By using 'Class._attr' style to access variables instead @prop function calls to Class.private variables we stay away from function overhead that is expensive. Nothing is truly private in python anyways so I do not find much benefit in burying things with the '.*' syntax.

Of course the rule is always never to pre-optimize, but chess may an exception due to the exponential blowup of moves. In any case, I think it is a good library which is why I am taking the time to explain this. If the move generation is fast enough it should be possible to write a simple eval function for alpha-beta search.

You can see the fork at https://github.com/nate1001/python-chess.

Regards, Nate.

niklasf commented 11 years ago

Hi Nate,

so I finally get to answer this properly - after three months.

python-chess was never intended to be used for high-perfomance chess engine types of things, but rather maybe for the GUI side of a chess program or simple move validation. Still, even for my purposes, it is pretty slow - even your fork, which makes some nice improvements.

I have rewritten the parts that were critical to me in C++, which makes it quite a bit faster (about 50 times on my machine): https://github.com/niklasf/libchess. After building it can be used from Python as well as from C++. I plan to add more, especially PGN reading.

Cheers, Niklas

niklasf commented 11 years ago

libchess works quite well. I've merged it into this repo. Once I am done with other things I'll consider using bitboards on the C++ side for extra performance.

It's no longer pure python, but for my project it was already worth it.