How to set VariantPath UCI option?

[gbtami]

While experimenting with local analaysis in pychess I noticed that analysis of user defined variants needs to set VariantPath. But sending window.fsf.postMessage('setoption name VariantPath value variants.ini'); in https://github.com/gbtami/pychess-variants/blob/master/templates/analysis.html#L49 doesn't seem to work. Is it possible that stockfish.wasm code needs some code using https://emscripten.org/docs/api_reference/Filesystem-API.html or am I missing something here?

[ianfab]

If I understand correctly, the reading itself perhaps should work, but the INI file needs to be packaged at compile time https://emscripten.org/docs/porting/files/packaging_files.html.

[gbtami]

I tried to build stockfish.wasm with --preload-file and now with --embed-file options also. No success so far :( With latest pychess commit https://github.com/gbtami/pychess-variants/commit/6ea9ae5087fda0569132df17ac2356cabaf3540b it produces Unable to open file variants.ini (This is the --embed-file version.)

[gbtami]

Notice that --preload-file version would be preferable to use, to let users change stockfish.data (variants.ini). Btw first use of Fairy stockfish.wasm already appeared :) http://www.xichess.com

[msabwat]

Hi ! Cool project ! I think you need to ship the .data file too. if you only ship .wasm/.js files emscripten's glue code won't find the file. I will let you know when I test it.

[msabwat]

After doing some tests today, the problem is not that the data is not here, but that the emscripten FS module does not find it. attempting to read the file in main.cpp/variant.cpp:

console.log(FS.readFile("variants.ini",{ encoding: 'utf8' }));

results in :

Uncaught (in promise) 
ErrnoError {node: undefined, errno: 44, message: "FS error", setErrno: ƒ}
[11:53 PM]

not sure yet if it's an emscripten bug, or if there is another issue. I could read the file with a (maybe too) reduced test case.

[ianfab]

The file embedding seems to work fine, so I suspect that the issue might be in how SF reads the file. The emscripten documentation on the filesystem API https://emscripten.org/docs/getting_started/Tutorial.html#tutorial-files only mentions fopen but not ifstream, so maybe the file reading would need to be converted to fopen, since -s FORCE_FILESYSTEM=1 did not help either. However, if this is the case, it might be easier to use an approach like https://github.com/ianfab/Fairy-Stockfish/commit/4e698c8f5cc51ea297ce60c0f07852589fdfa5af to be able to pass the file content at runtime, which would have the additional advantage that the variants.ini does not need to be known at compile time. The only issue with that approach is that a suitable delimiter for EOF needs to be chosen to avoid accidentally stop reading the string (and thus interpreting the rest of it as regular UCI commands).

[bmacho]

Is there a command to pass the content of the variants.ini file instead of its path?

[ianfab]

No, currently not. As mentioned this would require something like https://github.com/ianfab/Fairy-Stockfish/commit/4e698c8f5cc51ea297ce60c0f07852589fdfa5af.

[gbtami]

I try to merge https://github.com/ianfab/Fairy-Stockfish/commit/4e698c8f5cc51ea297ce60c0f07852589fdfa5af to current nnue branch.

diff --cc src/uci.cpp
index aa07bfb9,aef3fd1a..00000000
--- a/src/uci.cpp
+++ b/src/uci.cpp
@@@ -253,10 -230,21 +253,24 @@@ namespace 
    void load(istringstream& is) {

      string token;
-     std::getline(is >> std::ws, token);
-     std::size_t end = token.find_last_not_of(' ');
-     if (end != std::string::npos)
-         Options["VariantPath"] = token.erase(end + 1);
+     if (is >> token && token != "\n")
 -        Options["VariantPath"] = token;
++        std::getline(is >> std::ws, token);
++        std::size_t end = token.find_last_not_of(' ');
++        if (end != std::string::npos)
++            Options["VariantPath"] = token.erase(end + 1);
+     else
+     {
+         stringstream ss;
+         // Parse till double empty line
+         while (std::getline(cin, token))
+         {
+             ss << token << std::endl;
+             if (token == "" && cin.peek() == '\n' && cin.ignore())
+                 break;
+         }
+         variants.parse_istream<false>(ss);
+         Options["UCI_Variant"].set_combo(variants.get_keys());
+     }
    }

    // check() is called when engine receives the "check" command.

In pychess analysisCtrl.ts I added

        if (line.includes('uciok')) {
            window.fsf.postMessage('load ' + variantsIni);
        }

Result:

uci [chess:44:21](http://127.0.0.1:8080/analysis/chess)
Fairy-Stockfish [commit: b693b7de, upstream: , emscripten: 3.1.7] LB by Fabian Fichter [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
Unable to open file variants.ini [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
id name Fairy-Stockfish [commit: b693b7de, upstream: , emscripten: 3.1.7] LB [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
id author Fabian Fichter [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
<empty string> [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name Protocol type combo default uci var uci var usi var ucci var ucicyclone var xboard [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name Debug Log File type string default [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name Threads type spin default 1 min 1 max 512 [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name Hash type spin default 16 min 1 max 33554432 [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name Clear Hash type button [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name Ponder type check default false [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name MultiPV type spin default 1 min 1 max 500 [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name Skill Level type spin default 20 min -20 max 20 [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name Move Overhead type spin default 10 min 0 max 5000 [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name Slow Mover type spin default 100 min 10 max 1000 [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name nodestime type spin default 0 min 0 max 10000 [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name UCI_Chess960 type check default false [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name UCI_Variant type combo default chess var 3check var 5check var ai-wok var almost var amazon var antichess var armageddon var asean var ataxx var atomic var breakthrough var bughouse var cambodian var capablanca var capahouse var caparandom var centaur var chancellor var chaturanga var chess var chessgi var chigorin var clobber var clobber10 var codrus var coregal var courier var crazyhouse var dobutsu var embassy var euroshogi var extinction var fairy var fischerandom var gardner var giveaway var gorogoro var gothic var grand var grasshopper var hoppelpoppel var horde var janggi var janggicasual var janggimodern var janggitraditional var janus var jesonmor var judkins var karouk var kinglet var kingofthehill var knightmate var koedem var kyotoshogi var loop var losalamos var losers var makpong var makruk var manchu var micro var mini var minishogi var minixiangqi var modern var newzealand var nightrider var nocastle var nocheckatomic var normal var okisakishogi var opulent var placement var pocketknight var racingkings var seirawan var shako var shatar var shatranj var shogi var shoshogi var shouse var sittuyin var suicide var supply var tencubed var threekings var torishogi var xiangqi var yarishogi [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name UCI_AnalyseMode type check default false [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name UCI_LimitStrength type check default false [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name UCI_Elo type spin default 1350 min 500 max 2850 [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name UCI_ShowWDL type check default false [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name SyzygyPath type string default <empty> [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name SyzygyProbeDepth type spin default 1 min 1 max 100 [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name Syzygy50MoveRule type check default true [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name SyzygyProbeLimit type spin default 7 min 0 max 7 [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name Use NNUE type check default true [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name EvalFile type string default <empty> [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name TsumeMode type check default false [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name VariantPath type string default <empty> [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
option name usemillisec type check default true [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
uciok [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
Unable to open file Hybrid variant of Grand-chess and crazyhouse, using Grand-chess as a template [analysisCtrl.ts:521:16](http://127.0.0.1:8080/client/analysisCtrl.ts)
​```

I feel `(is >> token && token != "\n")` resolves to True, but my C++ is almost zero, so I have 0 idea how to solve this.

[ianfab]

        if (line.includes('uciok')) {
            window.fsf.postMessage('load ' + variantsIni);
        }

Assuming that variantIni contains the the content of the variants.ini but no leading linebreak, a linebreak between load and the variants.ini is missing. The pattern I assumed in that commit to distinguish it from a normal load filepath command is:

• load
• linebreak \n (start of file)
• content of variants.ini without any double linebreak
• double linebreak \n\n (end of file)

If we keep this approach it likely will make sense to use a more unique end of file marker to avoid accidental termination of the file. Because once it stops reading it as a file, it continues interpreting every line as a normal command, which could be quite dangerous.

[gbtami]

Thx, it works now. Do you have any suggestion for a more unique end of file marker?

[ianfab]

I think it might make sense to use an existing pattern like heredocs, which leaves up the EOF marker to the user

load <<EOF
[variantxyz]
...
EOF

[ianfab]

So I would propose something like https://github.com/ianfab/Fairy-Stockfish/commit/37e9d922310cf9ba48f25d165e88c474e2758e43

[gbtami]

My only problem is that I have absolutely no idea how to send EOF with window.fsf.postMessage() in javascript :(

[ianfab]

EOF here is just a literal string, no special character. You can specify any string (not containing spaces) as a separator, whether it is EOF or something else, just something that should not be contained in your data, i.e., the variant config. So it could literally also look like

load <<readuntilthisarbitraryseparationstring
[variantxyz]
...
readuntilthisarbitraryseparationstring

[gbtami]

Thx! This helped :) Seems it works now https://pychess-variants-pr-657.herokuapp.com/analysis/

[ianfab]

https://github.com/ianfab/Fairy-Stockfish/pull/455 is merged to Fairy-SF now. With the next merge to this fork, likely once the two other open PRs are merged, it will be in the WASM fork as well.