engine test suite format : bm and am ?

[tissatussa]

to test several engines, i use test suites with .epd files, with your nice little script /tools/analyse.py lately i found this puzzle :

5k2/7R/4P2p/5K2/p1r2P1p/8/8/8 b - - bm h3; am h5; id "LCTII.FIN.04"; c0 "Karpov - Deep Thought, Analyse 1990";

it took me a while to figure out what 'am' means : Avoid Move. indeed, some positions are traps : the engine may give any move as best, but not that am move .. here i see the code line contains bm AND am !?

What does it mean ? Should a -1 be given, like bm is positive 1 ?

Is the code line legal ? This is the only one i encountered having both bm and am .. can anyone point me to the syntax definition ?

[jdart1]

"am" is pretty common and is used in the Arasan test suite for example. It means that any move except the "am" move(s) should be credited as a correct solution. I don't think the EPD standard forbids having am and bm together, but in that case really only the bm field should matter: any other move should not be credited as a correct solution.

[tissatussa]

thanks !

for another question i could start a new Issue but it's related : not all UCI engines have a proper output and then your script fails .. that's OK, the engine programmer should properly implement the UCI protocol .. here's a typical log concerning my question :

DEBUG 12/14/202103:23:19 AM Using selector: EpollSelector
DEBUG 12/14/202103:23:19 AM Using PollingChildWatcher
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: Connection made
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: << uci
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> Ippolit2 0.9
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> PREFETCH
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> POPCNT
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> id name Ippolit2 version 0.9
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> id author John Smith
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> option name Hash type spin min 1 max 65536 default 256
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> option name PawnsHash type spin min 1 max 1024 default 32
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> option name PVHash type spin min 1 max 1024 default 1
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> option name MultiThread type check default false
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> option name Threads type spin min 1 max 16 default 1
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> option name Ponder type check default false
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> option name MultiPV type spin min 1 max 250 default 1
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> uciok
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: << setoption name Hash value 64
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: << setoption name MultiPV value 3
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: << ucinewgame
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: << isready
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> info string Optional Hash 64
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> info string Optional MultiPV 3
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> readyok
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: << position fen 5k2/7R/4P2p/5K2/p1r2P1p/8/8/8 b - - 0 1
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: << go movetime 10000
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> info depth 2
INFO 12/14/202103:23:19 AM @@ dInfo: {'string': 'Optional Hash 64'}
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> info depth 3
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> info depth 4
INFO 12/14/202103:23:19 AM @@ dInfo: {'string': 'Optional MultiPV 3'}
INFO 12/14/202103:23:19 AM @@ dInfo: {'depth': 2}
INFO 12/14/202103:23:19 AM @@ dInfo: {'depth': 3}
INFO 12/14/202103:23:19 AM @@ dInfo: {'depth': 4}
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> info depth 5
INFO 12/14/202103:23:19 AM @@ dInfo: {'depth': 5}
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> info depth 6
INFO 12/14/202103:23:19 AM @@ dInfo: {'depth': 6}
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> info depth 7
INFO 12/14/202103:23:19 AM @@ dInfo: {'depth': 7}
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> info depth 8
INFO 12/14/202103:23:19 AM @@ dInfo: {'depth': 8}
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> hash -3208363311884860365
WARNING 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: Unexpected engine output: 'hash -3208363311884860365'
DEBUG 12/14/202103:23:19 AM <UciProtocol (pid=67852)>: >> bestmove h6h5 ponder h7h8

this engine does not output any pv lines, only a bestmove .. i use this code piece to analyse a position :

with engine.analysis(
  limit   = chess.engine.Limit(10),
  multipv = 3,
  board   = myboard
) as analysis:
  for dInfo in analysis:
    [..somecode..]

but dInfo does not contain the string bestmove h6h5 ponder h7h8 .. i read other functions exist besides engine.analysis(), many are more complex .. Btw. i use logger with DEBUG level to create the log .. i tried other levels, but they seem to give less info ?

[jdart1]

The script does assume PV output is available. I don't think UCI mandates that: the output of "Info" is not well specified. It is up to the engine what it wants to output. Most do send the pv. "bestmove" however is required to follow every "go" command, once the search completes.