Leela Z integration ideas

[bochen2027]

Basically now that LeelaZ had gone "pro" and even the raw network (1 playout) is a good 4dan (using almost no CPU, doesn't need a GPU, and instant speed of play) I think there should be greater integration between Leela Zero and that of the awesome Sabaki! Leela Z is the best free open source zero bot and Sabaki is the best free open source Go GUI, a marriage of the two would be a match truly made in Go heaven. Yes, I know there is a LeelaSabaki, but that is geared towards the old version of Leela 11 which is already obsolete in light of the powerful Leela Zero that has way surpassed it's old sister program!

Unlike Leela 11, the much stronger Leela Zero doesn't have a native GUI, and its developer actively encourages people to use Sabaki as the front end GUI for the program!

We all know there is something called Lizzie, I spoke with the dev on github before and he would like for it to be on electron but he only knows java and doesn't have the time. Mainly, I speak for most people when I say that the greatest thing about Lizzie GUI is that it instantly shows in real-time the "heatmap" of LeelaZero (LZ) but also the percentages, the playout #, and the color coded (shades of Green to Red etc) which gives a great "at a glance" of the situation and of what the AI is thinking etc...

Currently Leela Zero is engine only, without a GUI, and the network weights come seperate. It has a high barieer of entry for newcomers because unlike the GUI version of Leela 11 it isn't a simple click install and play. Their website is also not very user friendly, and gives a lot of instructions on how to compile this or that, which is a turnoff for most casual Go players. I think the developers of LeelaZero and that of Sabaki could come together and agree to package their products together so that there is a simple but effective "one" download solution. This way, Sabaki will come with a default engine (Leela zero) but can attach other engines, and Leela Zero will come by default with a GUI (Sabaki)... Other forms of partnership if perhaps both developers agreeing to modify code of their end products to each respectively make integration of the two products easier and also to bring added functionality and value to the intermarriage of LeelaZero and Sabaki that otherwise would be difficult if not impossible when one sided.

https://github.com/featurecat/lizzie/raw/master/screen.png

So to start I propose a mapping like suches:

1) Spacebar Toggles on/off the "heatmap", and unlike the simple red heatmap of Leela 11, copy the more advanced "heatmap" of the Lizzie GUI where it shows in numbers the percentage of winrate, in numbers the total playouts for that move already calculted in realtime , and in spectrum of color from green to shades of red of all moves that the LZ game engine has considered in its analysis.. This requires a CODE modification in Leela Zero. The developer of Lizzie modified the Leela Zero code and included it within his Lizzie GUI, but a more streamlined way is to have Sabaki devs work directly with Leela Zero devs to encourage them to agree to the slight modifications themselves on the release branch and so it will be far easier for Sabaki to accomplish this sort of advanced heatmap readout that Lizzie has already done.

2) ALT (hold down) when combined with mouseover of a intersection point (legal move position) gives the predicted NEXT move of the LZ go AI pretending the player (human) already made the move at that location. For example, in my playing against the LZ bot I have to "undo" a lot and it takes a lot of time constantly having to go back. It would be make it much easier if there was an option when mousing over a legal potential move, instead of playing and clicking, I can just mouse over and hold down ALT on the keyboard and it will automatically show me what is the NEXT move of the LZ AI. This way I can rapidly play "what if" and explore the possibilites in a much more efficient manner without having to constantly undo and go back and whatnot. The longer I hold down the "ALT" at a particular location the more playouts the LZ AI will conduct and thus the actual place the LZ AI "next move" will land depends on how much time I gave it to calculate and it may dynamically shift, or if I shift my mouse when holding down ALT then my mouseover of a different location of course will change the entire outcome of the location of the "next move" of LZ. But everything would be very fluid and dynamic without need to "undo" or go back etc.

3) CTRL (controlk) hold down when combined with mouseover of a particular position explores the branching playout variation that stems from that position. This functionality will make In-game dynmaic real-time analysis much more better! Instead of reading long logs, I can explore infinite variations of playouts and real-time selectively tune those playout variations merely by hitting a Control key when mousing over. For example, lets say that I toggled spacebar ON, ("heatmap",) and I'm playing against LZ and its my turn to move, it already shows me chooses for what my next move could be, but lets say instead of all the listed moves I wanted to make a move that the AI didn't predict, and so I mouse over a legal move location, hit the CTRL key and hold it down, then the AI on the backend calculates the variations of the playouts as if I had already made that move (I had not!) in the game and then in opaque coloring plays out the playout in front of my eyes (with each move numbered) for say a preset configuration length (say 10 moves) so basically at any time in the game, regardless of whose turn it is, I can instantly mouse over at any legal move position on the board and it will show me the sequence of playouts or variations for that move for the next Nth amount of moves, and if there is more than one variation stemming from that position, I just have to scroll my mouse wheel up or down to cycle through the different variations all stemming from that one singular position! And as I cycle with the mousewhell through the different variations per that one move location that I'm moused over while holding down CTRL key, it will show me the next Nth moves per each variation, in the blink of an eye, all nicely numbers on the board, dynamically in real time... and all I have to do is move the mouse to a different location and it starts all over again.... this is a great way to instantly explore countless possibilites without the hassle of reading long logs and going back and forth,,, its how to visually deal with information overload and will massivly help beginner players get strong and quickly and effectively analysis their games!

4. Support for importing Leela Zero logs into Sabaki for analysis (in addition to loading sgf, allow us to load the log file from Leela Zero and have Sabaki meaningfully display and intrepret it with branching variation shown... ... No one wants to read 80+ move sequences and long logs like this: https://ia601502.us.archive.org/2/items/LeelaZeroGod/move78_logs.txt

Sabaki could be the perfect way to read these logs by natively importing these logs and having the branching variations mapped out and displayed in the Sabaki "Show Game Tree" and allowing the end user to graphically understanding the analysis of the game in a much easier way!

5. TCP/IP and BLOCKCHAIN support : Enable the Sabaki client to connect to the outside world via IPv4 and IPv6 and integrate Blockchain support into the fabric of Sabaki with its own distributed ledger system to keep track of scored games, to do foolproof "timestamping" verifications, sprout the first "Go" crypto currency, including secure end to end encrypted chat and messaging within Sabaki client (like bitmessage) using the underlying blockchain and TCP/IP, with support for running over the TOR for better enhanced privacy, and by default the Sabaki client can do pgp-like features like digitalitty "signing" played games to make them more official etc. (p2p, proof of work concept etc) With an established blockchain and crypto currency Sabaki can even add a commercial layer to it, to be the marketplace of bring Go teachers and Go students together via the online internet and taking a slice of the pie when an exchange of money occurs, or say with encryption it can secure sessions with a key and also get into what patreon is doing for the likes of Nick Sibicky and other Go broadcasters on Patreon.

So this will allow Sabaki clients to directly connect to other Sabaki clients to play games via the LAN/ internet, and by using blockchain tech with a federated metaserver it can also take out the middle man (KGS/OGS/etc) and basically allow clients running Sabaki to directly find and play games and matches all within the Sabaki GUI. Coupled with Sabaki web version and perhaps also in the future an android and/or tablet version, this can entirely take the place of OGS/KGS.

Using multicast IPv6, or other ways, popular games like pro games can be broadcasted and watched by thousands of players via the Sabaki client, right within the client itself without having to rely on some other third party online hosting service.

Think like in terms of ONLINE/OFFLINE mode, PRESENTER/REVIEWER mode, CO-PLAY MODE, SPECTATOR mode, TRAINING SELF-PLAY mode... etc

6. TABBED instances. Have like Firefox, where using a tab will allow multiple instances/games of Sabaki running as tabs. So they can quickly be cycled through and have the default tab show mini games at a glance so from one pane of view you can see all the games you have open in all the different games and their real time status and more better and quickly find and enter and navigate to the tab instance of your choice. This will be handy when you have a mix of different games, like if you were reviewing a game in one tab, playing a game online with another player in a second tab, and doing an AI vs AI game in a third tab, and playing a game against an AI in a fourth tab, and watching a pro game being broadcasted in a fifth tab, and so on and so forth.

7. Support for native screenshot and screenrecording and in-game session recording capabilities, with a lossless compression CODEC for producing quality screen/session recording at very small file sizes. Imgur integration to automatically upload screenshots to imgur, or perhaps vimeo, dropbox, youtube, twitch integration to more easily upload videos to online streaming services.
   Sabaki native in-game "blackbox" recording will be much smaller in size like sgf sized since it doesn't have to render out anything, but basically it will show everything that happened including capturing mouse movements and mouse clicks and etc so make a extensible bversion of SGF whereby the "recording" itself is embedded into the sgf and then later on it allows the player to seemlessly replay all games as if watching it for the first time live, and supports pause, rewind, and variable rate of playback speeds.

8. LeelaWatcher replacer. Essentially there is no need for a separate tool to watch LeelaZero do its self playing training games when in fact Sabaki can be made to do the exact same function. Esp. with the advent of TABBED instances, this would work great with those who are running multiple different instances of LeelaZero training self playing games at the same time, the user is now able to watch all of his instances of Leela Zero training within the single pane of glass from Sabaki's support of tabbed instances and showing mini-preview on the default "homepage" of Sabaki client.

9. Open API so that others can create extensions and plugins to sabaki to allow for example a CGOS watcher so now we don't have to have some crappy java tool to watch cgos games in real time etc; infact with a robust API and extension/plugin support, someone else can create the LeelaWatcher tool instead of the main Sabaki dev.

10. Support for TRUINE brain mode. Some Chess GUI allow attaching three engines, and combining input from all three and choosing a master instance to arbitrate, essentially this gives a broader range of ideas that one engine might miss. For example I'd like to be able to have three brains, AQ, LeelaZ and some other bot maybe Minigo if it ever gets stronger, and then doing each move the engines will make variations suggestions but its up to the master engine designated within Sabaki that ultimately makes the final call...

11. Drag and drop SGF and Leela Zero log files. I'd like to be able to instantly drag and drop sgf files into sabaki and also native support for Leela Zero log files (with full variation branching support) so that in conjunction with the multiple tabbed instances support, I can rapidly drop lots of games and lots of log files into Sabaki with ease and then tab between them for a more effective game reviewing sessions...

12. Allow full featured pencil/drawing overlays with scalable network connectivtiy, combined with the aforementioned multiplayer /internet connected modes this will allow real-time group reviews of games (think etherpad or Google Wave), esp if there is some form of native video or microphone support... With the ability to natively record and save sessions, this also allows people to post/host their reviews of games, perhaps even layer a commercial aspect by password protecting access to certain games and game reviews and this affords a global teacher /student , reviewer/reviewee system and Sabaki could take a slice of the ecosystem when value exchanges hands especially when using its native blockchain and its Go cryptocurrency...

13. People who don't want to donate money can donate cycles instead. For example, whenever end user has Sabaki client open, it will ask permission to use up to say 10% of CPU/GPU resources to "mine" for Go cryptocurrency or in other ways to help with other efforts like towards the LeelaZero training effort... Even create a screensaver mode whereby when computer is idle it will start a selfplaying session of Leela Zero and help LeelaZero train in the background and/or do some Go crypto mining in the background when its not doing anything else.. Instead of OGS asking for money in exchange of blocking Google adwords on their website, the way to go in the future is for trading idle cycles for go crypto currency. The paypal donation/ and Google adwords/adsense model is a thing of the past. In this day and age everyone is using ad blockers and no one is donating via paypal anymore.

14. Have a partnership with leelazero, in exchange for putting a mechanism embedded within each download of Sabaki that prompts the end user to give permission to donate idle time to the LeelaZero training effort, that LeelaZero agrees to prepackage all downloads of LeelaZero with Sabaki baked in as a one click download/install /play. Sabaki will give LZ more exposure and potentially more clients joining the training effort whereby making LZ that much stronger and if LZ becomes the most popular default Go engine then by coming prepackaged with Sabaki, Sabaki will cement its position as the most widely adopted Go GUI platform.

https://github.com/gcp/leela-zero/pull/704

[alreadydone]

There's also https://github.com/SabakiHQ/LeelaSabaki, though maybe not very tailored to Leela Zero.

[alreadydone]

Hi Yichuan, I see your comment in an email; it's not showing here but I still want to reply.

I said LeelaSabaki is not tailored because it also work for the original Leela, but Leela Zero doesn't quite work the same way, so it was not created for the specific purpose of Leela Zero (but since LZ has become the strongest open-source bot this is definitely worth doing, as has been taken up by the developers of Lizzie). This is pure speculation though; I am not at this moment an avid user of (Leela)Sabaki and launched it maybe only three times. I do think Sabaki has a more elegant UI. It turns out the original developer of Lizzie modified the source (see https://github.com/featurecat/lizzie/issues/24) of leelaz (the GTP engine) to get the output he need to show analysis on board; people have been using the UI to analyze pro games with pro or AI in real time. If you didn't follow the development and growth of Leela Zero, here are some possibly interesting resources: Github issues: https://github.com/gcp/leela-zero/issues reddit forum: https://www.reddit.com/r/cbaduk/ Elo rating graph: http://zero.sjeng.org/ Leela Zero QQ group: 726658329 Lizzie developers' QQ group: 638934756 discord: https://discordapp.com/invite/cA2dekV Friendly link: LCZero, the chess port of Leela Zero http://lczero.org/

[bochen2027]

There is a hidden comment?!

[yishn]

I'm sorry, I deleted my comment, because I've changed my mind about what I said. Don't worry about it...

[bochen2027]

Thanks for the title change to more accurately reflect my post, I editted my post. Hope you can consider some of the suggestions and ideas. Sabaki is great but it can be made a lot better too! Leela Zero is hitting the Go world by storm and it would be great if better integration by default was possible plus other innovative features that Sabaki can do with in parntership of Leela Zero

[HenryHo2006]

I'm sure that LeelaZero will become the most powerful and most popular Go software engine. The best choice for the most powerful GUI software is to adapt it.

[alreadydone]

The OP's blockchain proposal seems much more far-reaching than mine ... Anyway, if anyone is interested, here is my proposal.

[yishn]

Oh wow, I didn't realize @hydrogenpi has updated their original post to such proportions... We're definitely not going to implement some kind of blockchain inside Sabaki. These are all great ideas, but Sabaki is an SGF editor after all, that's able to play some engines, not some kind of "eierlegende Wollmilchsau".

[bochen2027]

@yishn what about a touranment mode? Right now Leela Zero has hundreds of networks now that not counting the unofficial 256*20 networks that are coming up left and right. Let says I want to scedule 100 games and leave my computer on overnight, I can do this in any old Chess engine like Shredder but in the Sabaki this isn't an option and I have to do one game at a time?

Can you make this change?

Lizzie is innovating faster with features tailored to LeelaZero and with the rise of LeelaZero its catching mindshare as many people are already using it for game reviewing etc personally I do not like java and I think Sabaki is more refined, less buggy and more robust, however Lizzie is iterating very fast.

[yishn]

Can you share some screenshots in Shredder how the tournament scheduling workflow works?

Please keep in mind that this is no competition; open source thrives because of the diversity.

[bochen2027]

In my opinion the open vs closed (source) debate exists within the context of the larger dynamics of evolutionary strategies against the backdrop of entropy maximization / 2nd law of thermodynamics. Even within the open source arena, there always exists competition in the form of mindshare and whatnot.

if Lizzie catches up with Sabaki in terms of feature parity and offers customized features for Leela that Sabaki doesn't, what is to prevent a mass exodus? lets consider a hypothetical worse case scenario, that leela becomes the top open source bot and basically in the future there is a consolidation of go bots and Leela becomes the default bot, the one used just about by everyone. Then Lizzie will take over too, since it is so tightly integrated with the LZ. There is an oppurtnity for a massive windfall for Lizzie since its riding the rising tide of Leela Z.

In terms of low hanging fruit, the one single most important thing you can do right now is to incorporate some sort of color analysis mode like what Lizzie has right now. This will stop the bleeding and give you a chance to recup before its too late and Lizzie takes the lead.

As most casual go players are only just now coming onto the LZ scene, its quite possible whatever GUI they first use is the one they will stick with long term. Right now for LZ, it could be argued that Lizzie is on par or in some ways even better than Sabaki when considering how closely tailored it is to the mother program. Sabaki can find a way to do the same by creating a wrapper around LZ without having to do a code change of LZ (Lizzie distributes LZ with modified source code changes so it can output the stats during its analysis to be as inputs to the color mapping modes etc) and if you incorporate other features like the "ALT, CTRL, Spacebar" that i mentioned above in conjunction with say the ability to drag/drop LZ log files directly into a (tabbed or nontabbed) Sabaki window to show all the game tree + variations + color analysis /heatmap modes then it will stop the threat of Lizzie. Again, as already stated, I prefer Sabaki to Lizzie for many reasons, including its more refined, been in development far longer, has a much larger install base, more mindshare, is more robust, allows other game engines, and doesn't use the antiquated Java. But there is no denying that Lizzie has many attractive features of LZ that Sabaki cannot currently match.

I would prefer not to have to use two different programs of Go Gui and having to switch back and forth, context switching is not effective and neither is market fragmentation.

I'm rooting for Sabaki and want it to be the dominant supreme Go Gui out there.

I will upload screenshots this evening.

https://www.youtube.com/watch?v=ZgDjHJ9IYU0

https://www.youtube.com/watch?v=Pg7cgynSzHc

[Cabu]

I am in a process to propose a patch to Leela Zero implementing the changes for Lizzie as a command line parameter. That way other GUI like sabaki could take advantage of lizzie output without bothering to have their own version of the engine.

[Cabu]

Done: https://github.com/gcp/leela-zero/pull/1240

[Cabu]

I really think that people would like something more than just an SGF editor, but something were thay can play games, have real-time analysis (eg: https://github.com/featurecat/lizzie), and doing review/post-mortem analysis of games (eg: https://github.com/pnprog/goreviewpartner), and even playing through it on online go servers (EG: KGS, OGS & al...).A "eierlegende Wollmilchsau" as you say it.

I think that a lot of people think that Sabaki is a good start to integrate the needed features to this goal because they really like the way the interface is done. If Sabaki would limit itself to were it is now, it would lose its lead for other probably inferior GUI :(

Now I understand that integrating such features while keeping the Sabaki look & feel and ease of use would be a challenge in itself.

[ghost]

@Cabu I've done some commits on my branch of leelaSabaki and Sabaki so that it can be used to review games. it doesn't use the 'live' leelaz patch, but gets the pondering variations and genmove variations to update the game tree, add A,B,C,... labels.

If I'm playing black against leelaz the commands would be something like : '--black --labels --limitdepth --flat' leelaz leelazargs

When the game is finished leelaz's evaluation/variations for black are in the gametree for review. Very similar to goreviewpartner's live analysis function. GRP has some advantages such as the winrate graph, and better able to see variations. LeelaSabaki has some advantages such as easier navigation of the gametree. I think live lizzie analysis is especially usefull for live commentating on games.

[bochen2027]

https://github.com/SabakiHQ/Sabaki/issues/387

[tjdcs]

Where are the devs chatting about this? I think the user desire can mostly be summarized by saying "I want to use LeelaZ to help analyze and review my games"

LeelaSabaki looks like a fair enough start. Why isn't that repo just a branch of this one? Seems like the two have probably diverged a bit.

If there are people working on this are there any areas of the code that need more time / attention? I may be able to help. I'm mostly a vue / typescript developer right now but not sure where to dive in and don't want to duplicate someone else's effort.

[yishn]

LeelaSabaki is just a thin layer between Leela and Sabaki, not a complete fork of Sabaki.

Right now, I've implemented analysis mode in the current master of Sabaki that's compatible with the next branch of Leela Zero. Give it a test if you want. I'm hoping to implement a winrate graph as well. These features will be in the next major release of Sabaki, after a new version of Leela Zero has been released. The use of LeelaSabaki will be deprecated then as well.

[tjdcs]

I'll take a look at the latest master then, maybe this weekend.