Can one view the different EGN moves for perft?

[andersfylling]

I'm trying to debug my own engine using perft, but the fact that i have 2020 promotions too many given a certain FEN and depth, doesn't help me too much.

So I was curious if this engine supports printing out all the EGN moves during perft? Ofcourse only when you count the number of moves as well.

eg.:

a5d5: 89
a5e5: 89
a5f5: 89
a5g5: 89
a5h5: 89
a8a1: 252
a8a2: 256
a8a3: 260

[jniemann66]

Hi there. I feel your pain !! debugging perft errors is hard, and promotions are always a source of trouble.

When I was developing juddperft, I had a secret test mode that invoked an external perft engine to compare against. It's been a long time, and my memory is a bit rusty, but I might be able to re-enable it.

Can your engine take a FEN position from std input, and return its perft number to std output ?

If so, we can probably rig up my engine to test yours. Basically, it splits the given starting position, and when there is a disagreement on a particular node, it splits that node and repeats the process recursively until the offending position is found.

[andersfylling]

Oh nice, and yeah I just added perft so that it only spits out the node count. The syntax is a little weird tho: "perft DEPTH FEN " where * is the desired value. "perft DEPTH 5 FEN 8/2p5/3p4/KP5r/1R3p1k/8/4P1P1/8 w - -". Default depth is 5 and default FEN is a typical board setup.

Here's the branch with the perft support. https://bitbucket.org/sciencefyll/david/src/3d7eeb5c27a7465dc87ac1deb35b4c1fe09bee57/?at=MoveGenA1

However, and I just recently noticed this, I've written mine on linux. I haven't even tested if it compiles on Windows. Let alone how it behaves :/

I think I'll look a little more into your code and see if I can add cross platform support.

[jniemann66]

Yeah ok - funny - I was just looking at getting mine to compile on Linux and Mac hehe. (I will need to clean up all those __int64 types etc).

I just noticed that mine has a function in diagnostics.cpp: int PerftValidateWithExternal(const char* const pzFENString, int depth, __int64 value)

So, the way it works is that it invokes some external perft program with a position, depth, and expected perft value. The external perft program is expected to return EXIT_SUCCESS if it agrees. (My engine assumes that the external perft engine always has the correct answer).

the path is currently set with a global variable (in Diagnostics.h): const char PerftValidatorPath[] = "c:\bin\PerftValidate.exe";

I can probably clean all that up to use the parser, so you could interactively enter a command something like this:

testexternal <path-to-external-binary> <depth>

... and it would recursively validate the current position against the external engine.

Let me do some work on it, and see what I can do ...

[andersfylling]

I did start to do a little of the ifdef _WIN32 etc.: https://pastebin.com/MM1VA4sC and https://pastebin.com/eaB6qYre

What I noticed is: unsigned __int32, can be replaced with uint32_tas of c++11 http://en.cppreference.com/w/cpp/types/integer

which mean you also have uint64_t, int32_t, and ull to replace i64 for integer literal suffix.

When it comes to teh headers such as intrin.h I have honestly no idea.. search.h:4: windows.h diagnostics.cpp: diagnostics.h

those were the only includes i had issues with, except the ones I have "dealt" with in the pastebin links.

[jniemann66]

Yeah - I'll have a look at it later tonight (in about 6 hours or so ...)

[jniemann66]

So, I replaced all the __int64's with int64_t's (and unsigned __int64's with uint64_t etc). Same for a whole lot of other integer types.

Then, I had to think about the 64-bit inititalizer suffixes (i64 is windows-specific). It is ll or LL (or ULL for unsigned) on other compilers. But, if I'm not mistaken, the only time I may need to explicitly specify LL is when there are large bit shifts on a small (32 bit) initializer, eg: int64_t x = 1LL << 56; So, I cleaned a lot of that up.

the intrin.h can just be replaced with x86intrin.h

Then, I ran into problems with microsoft's printf_s and friends. I also had to just comment-out some windows-specific stuff, including the timer macros.

So, I fixed all of that, but now, with clang, I'm getting this one compiler error:

object expression of non-scalar type 'ChessMove [128]' cannot be used in a pseudo-destructor expression

I haven't had time to work out what is going wrong with that one yet

[andersfylling]

yeah so c++11 supports ull, ll, etc as a standard. so you don't have to worry about that :) http://en.cppreference.com/w/cpp/language/integer_literal

I'd recommend using C++14 tho as they now support binary expressions as well: 0b00000001.

Can you push your changes into a experimental branch? Then I can take a look as well. Perhaps that is the nix branch? my mistake ^^,

[andersfylling]

I'm compiling it on my system now (linux 64, gcc 7.1, CMakeList 3.9) and I notice that I'm unable to use gets_s (winboard.cpp#113).

Can I suggest using the cpp way?:

std::string command;
std::getline(std::cin, command);
if (!command.empty()) {
  ....

then to get the char array as you use other places, if needed: command.c_str();

[jniemann66]

Yep - I just got too tired and ran out of time. I'm going to see if I can finish it off now ...

[jniemann66]

I got it to compile with GCC 5.4 It runs, but gives nonsense output at perft 7 Most likely the chosen integer types or their initializers (or both) are still wrong. Will need to do some more work on it tomorrow

[andersfylling]

This is from the latest nix pull with -lpthread and std=c++14:

CMakeFiles/juddperft.dir/search.cpp.o: In function `std::atomic<LeafEntry>::load(std::memory_order) const':
/usr/include/c++/7.2.0/atomic:250: undefined reference to `__atomic_load_16'
CMakeFiles/juddperft.dir/search.cpp.o: In function `std::atomic<PerftTableEntry>::load(std::memory_order) const':
/usr/include/c++/7.2.0/atomic:250: undefined reference to `__atomic_load_16'
CMakeFiles/juddperft.dir/search.cpp.o: In function `std::atomic<LeafEntry>::compare_exchange_weak(LeafEntry&, LeafEntry, std::memory_order, std::memory_order)':
/usr/include/c++/7.2.0/atomic:291: undefined reference to `__atomic_compare_exchange_16'
CMakeFiles/juddperft.dir/search.cpp.o: In function `std::atomic<PerftTableEntry>::compare_exchange_weak(PerftTableEntry&, PerftTableEntry, std::memory_order, std::memory_order)':
/usr/include/c++/7.2.0/atomic:291: undefined reference to `__atomic_compare_exchange_16'
collect2: error: ld returned 1 exit status
make[3]: *** [CMakeFiles/juddperft.dir/build.make:303: juddperft] Error 1
make[2]: *** [CMakeFiles/Makefile2:68: CMakeFiles/juddperft.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:80: CMakeFiles/juddperft.dir/rule] Error 2
make: *** [Makefile:118: juddperft] Error 2

[jniemann66]

you need the pthread and atomic libraries. (-pthread and -latomic)

I just use a raw gcc command like this: g++ -pthread -std=c++11 *.cpp -o ./juddperft-gcc -latomic -O3

[jniemann66]

... but it's still broken. I have to get some sleep and go to work in the morning :-)

[jniemann66]

OK - I did some more work on it. It's not quite there yet, but getting close ... Also, there is something weird happening with it's multithreading on Linux. (It keeps getting "stuck"; for now you can run on 1 thread only by entering cores 1)

I need to investigate it.

[jniemann66]

OK - so now, I have fixed all the issues with multithreading, and added the test-external command (inside juddperft)

Usage: text-external <path to external app> <depth>

This will issue the following system command for each test position: <external app> "<Fen String>" <depth> <perft value> external app is expected to terminate with EXIT_SUCCESS (0) if it agrees with the perft value for the given position.

It is up to you to get your engine to accept the 3 parameters (fen-string, depth, value) and return 0 if it agrees or any other value if it disagrees.

If you can't get your app to read the arguments in that syntax, you may also be able to write a shell script to interface to your app. Alternatively, you can change the PerftValidateWithExternal() function in diagnostics.cpp to make it do whatever you want.

Anyway, the way it finds perft bugs, is to divide the current position into all available moves, and run a perft at depth-1 for each move. If there is a disagreement in the perft score for one of those, it divides again from that position, and so on recursively until it gets down to a final position. It's a powerful way to track down where a bug is in an engine.

I hope this helps. Let me know if it's of any use to you.

PS - just re-reading your original question - you can also simply use the divide or dividefast commands inside juddperft.

eg

dividefast 6
Ng1-f3 ....................Perft 5: 5723523
Ng1-h3 ....................Perft 5: 4877234
Nb1-a3 ....................Perft 5: 4856835
Nb1-c3 ....................Perft 5: 5708064
 h2-h3 ....................Perft 5: 4463070
 h2-h4 ....................Perft 5: 5385554
 g2-g3 ....................Perft 5: 5346260
 g2-g4 ....................Perft 5: 5239875
 f2-f3 ....................Perft 5: 4404141
 f2-f4 ....................Perft 5: 4890429
 e2-e3 ....................Perft 5: 9726018
 e2-e4 ....................Perft 5: 9771632
 d2-d3 ....................Perft 5: 8073082
 d2-d4 ....................Perft 5: 8879566
 c2-c3 ....................Perft 5: 5417640
 c2-c4 ....................Perft 5: 5866666
 b2-b3 ....................Perft 5: 5310358
 b2-b4 ....................Perft 5: 5293555
 a2-a3 ....................Perft 5: 4463267
 a2-a4 ....................Perft 5: 5363555

Perft 6: 119060324
Time=16911 ms

Note: to set up the initial position , use the setboard command

Cheers, Judd

[andersfylling]

How exactly does it test each individual move?

Cause testing it at standard perft depth 1 tells me everything is incorrect. However debugging every move created by my own engine, I see that each move is correct. So I'm confused how yours validates this.

[jniemann66]

Hi - it doesn't test each individual move. It works using the "divide and conquer" approach to recursively hone-in on where the problem is. Lets say you a position for which the two engines disagree on perft 5. As an example, you could have something like this:

starting position: (we want to test test perft 5, white to move)
    make white move 1 : get perft 4 (ok)
    make white move 2 : get perft 4 (ok)
    make white move 3 : get perft 4 (PROBLEM ! engines disagree )
        make black move 1 : get perft 3 (ok)
        make black move 2 : get perft 3 (ok)
        make black move 3 : get perft 3 (ok)
        ...
        make black move 15: get perft 3 (PROBLEM ! engines disagree )
            make white move 1: get perft 2 (ok)
            make white move 2: get perft 2 (ok)
            ...
            make white move 5: get perft 2 (PROBLEM ! engines disagree )
                make black move 1 : get perft 1 (ok)
                ...
                make black move 7 : get perft 1 (PROBLEM ! engines disagree) <= this is where the problem is - one (or both !) of the engines generated wrong number of moves

maybe you could send me your results and I'll have a look

[andersfylling]

This is the correct format right? ./chess_ann_src "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1" 1 20 Cause here my engine returns 0.

Me engines source code can be found here: https://bitbucket.org/sciencefyll/david/src/204a56af811e835dc060c6b568d1cecb72dfbfe8/?at=MoveGenA1

must be branch MoveGenA1

 anders@arch-laptop: /home/anders/Projects/david/cmake-build-release/bin/chess_ann_src "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1" 1 20
 anders@arch-laptop: echo $?
0

But when I try this using juddperft:

test-external /home/anders/Projects/david/cmake-build-release/bin/chess_ann_src 1

Testing against external engine: /home/anders/Projects/david/cmake-build-release/bin/chess_ann_src to depth of 1
Position:

---------------------------------
| r | n | b | q | k | b | n | r |   Black can Castle
---------------------------------
| p | p | p | p | p | p | p | p |   Black can Castle Long
---------------------------------
|   |   |   |   |   |   |   |   |   White can Castle
---------------------------------
|   |   |   |   |   |   |   |   |   White can Castle Long
---------------------------------
|   |   |   |   |   |   |   |   |   
---------------------------------
|   |   |   |   |   |   |   |   |   
---------------------------------
| P | P | P | P | P | P | P | P |   material= 0
---------------------------------
| R | N | B | Q | K | B | N | R |   White to move
---------------------------------
Off we go ... 
Found Bad Position!

---------------------------------
| r | n | b | q | k | b | n | r |   Black can Castle
---------------------------------
| p | p | p | p | p | p | p | p |   Black can Castle Long
---------------------------------
|   |   |   |   |   |   |   |   |   White can Castle
---------------------------------
|   |   |   |   |   |   |   |   |   White can Castle Long
---------------------------------
|   |   |   |   |   |   |   |   |   
---------------------------------
|   |   |   |   |   |   |   |   |   
---------------------------------
| P | P | P | P | P | P | P | P |   material= 0
---------------------------------
| R | N | B | Q | K | B | N | R |   White to move
---------------------------------
Ng1-f3
Ng1-h3
Nb1-a3
Nb1-c3
 h2-h3
 h2-h4
 g2-g3
 g2-g4
 f2-f3
 f2-f4
 e2-e3
 e2-e4
 d2-d3
 d2-d4
 c2-c3
 c2-c4
 b2-b3
 b2-b4
 a2-a3
 a2-a4

[jniemann66]

Yes, the format is right, but test-external is not designed to be run at a depth of one. It is meant for high depths when you are looking for the "needle in the haystack" "found bad position" is just what it says when it reached a terminal (leaf) node -Maybe I should change the wording. If you run it from a depth greater than 1, and both engines agree on everything. it will never reach that state.

You originally said that you had "2020 promotions too many given a certain FEN and depth" What is that position ? and what is that depth ?

You should try the following in juddperft:

setboard position test-external /home/anders/Projects/david/cmake-build-release/bin/chess_ann_src depth

and if you do get "found bad position" , then what happened immediately before that should be a big clue as to what is wrong.

Good luck !

[andersfylling]

Alright, I assume I got it working now then ^^,

Here's some of the results, how exactly do you understand what's incorrect here. Right now I get the impression there's something wrong when black is in check, but I'm not certain.

https://pastebin.com/B5QwfPmw

Edit: It's actually kinda beautiful how much info this function spews out.

Edit2: MoveGen question. When a pawn is moved into the 8th rank, should that count as 1 move or 4 moves? since it can be promoted to 4 pieces, which in itself is 4 different choices.

Edit3: I understand everything now. Thank you!

[jniemann66]

Ok - first things first. I just did another commit, in which I cleaned-up the behavior when it reaches a terminal node. It now does the following (at depth 1) :

1. Prints "Reached Single Position!" to stdout
2. does a perft(1) from this position (ie number of legal moves for position) and checks whether external engine agrees.

Ok - I got your results - so now the detective work begins ! I'll have a look at it (after I have had a few strong coffees hehe)

[andersfylling]

Haha, thanks for the effort! But thanks to this I found the issue!

The way I deal with enpassant is rather primitive. And I use the value 0 whenever there is no enPassant. However whenever black is moving, this is converted to a bitboard with a set bit at index 0. so when theres a black pawn at index 9. it thinks theres an enemy at index 0 even though there isn't. Added a simple check and TADA!

[jniemann66]

Awesome - I'm super happy that my app was helpful (and not just an intellectual wank-fest that nobody cares about except me haha ).

You also helped me improve mine a bit - particularly since I finally got off my ass and compiled it for linux !!

Best of luck with your engine !