search

glinscott / leela-chess

**MOVED TO https://github.com/LeelaChessZero/leela-chess ** A chess adaption of GCP's Leela Zero
http://lczero.org
GNU General Public License v3.0
759 stars 301 forks source link

LCZero promotion, slogan, etc #120

Open CMCanavessi opened 6 years ago

CMCanavessi commented 6 years ago

Now that LCZero is training and gaining elo quite rapidly, I'd like to promote it in several forums but I was thinking we need something catchy to attract users so they can not only help, but also follow the progress. I have a couple of ideas.

1) have a long term goal, a big one, even if it's unrealistic. Just needs to sound cool, for example "aiming to beat Stockfish 3 by the end of 2018, will you miss being part of it?" or something like that. Stockfish 3 is 3128 elo in CCRL 40/4 single CPU rating list. You may think that it's completely crazy and out of reach, but that's exactly what we need for users to "bite" and help the project grow.

2) for short term goals, we should aim at classic very well known or famous engines, Fruit for example, though it's still way far ahead. I've been checking the rating lists and it's possible to come up with a list of actively developed engines in the whole range of strenght, from 1000 all up to Stockfish 9. It would be nice to have these kind of achievements like "Gen 25 is already stronger than Sabrina" (which is around 1300 elo, and its programmer is active on the forums". After that is achieved, we aim for TSCP (very well known simple engine) which is around 1700 elo. I think for ppl to understand progress it's better to say "now it's stronger than X" (X being a well known engine) than saying "now it's 2351 elo" which means you have to go look at the rating lists to see what the hell that means in term of relative strenght to other engines.

Other ideas welcome :)

PS: and a logo, we need a logo, urgent

jkiliani commented 6 years ago

I think aiming for Stockfish 3 by the end of the year is not completely out of reach. The AlphaZero paper showed that training a net to play chess at very high level takes fewer resources than training it to play high level Go, and look where Leela Zero is already after 4 months: It regularly beats professional players already.

If LCZero can get a similar level of compute as Leela Zero, it will grow very fast indeed. Leela Zero also showed that there's no need to restart once the neural network stops improving: You can simply train a larger size net on existing self-play games and keep going.

I also believe at some point the Stockfish community will be very interested in trying to include a strong ResNet from LCZero into Stockfish to replace or complement the quiescence search.

jhellis3 commented 6 years ago

Well, the strength of LCZ is going to be strongly hardware dependent initially. But assuming the proper hardware, I would expect LCZ to be the 4th strongest engine within 6 months.

If it is allowed to train vs conventional engines, I have little doubt it will smash them all by the end of the year.

Barring other AI entries, LCZ is going to quite popular very soon... it won't need promoting ;).

jjoshua2 commented 6 years ago

I think we should have the goal of beating SF8 by the end of the year, which is what AlphaGo achieved, and with the rate leela zero made, I think we have a good chance, especially if we do things like add TB play and adjudication.

109 was designed to get the list of engines to get excited about beating at different levels. I think the ones I called out are good ones, but yours as are well and we need them at different levels.

I think ali_tcec who does lots of good random logos in tcec chat could do something interesting and decent quickly.

CMCanavessi commented 6 years ago

Well, I didn't want to be THAT optimistic about beating SF8 already, but yeah, we can also say that.

That's a nice list there on #109 but they are all too strong, and they are too spaced out. I'll make a list "from 0 to heaven" with intervals of around 200 elo and post it here later. I will mix classic enignes and new, actively developed ones.

CMCanavessi commented 6 years ago

We could also put something like this https://3.bp.blogspot.com/-NyHN2-tzjdQ/ToCmOac1fkI/AAAAAAAABVE/oT1IY_AoE8k/s1600/homer_edison.jpg in the main website, edited with classic familiar chess engines at all strenghts. Would look cool at least

jjoshua2 commented 6 years ago

Logo ideas 1, 2 I like that race idea too

CMCanavessi commented 6 years ago

Here's some sort of checklist that we can update with the date it was achieved:

Achievement                            Date
+500 elo                              March 13, 2018
+1000 elo
Stronger than Sabrina
+1500 elo
Stronger than TSCP
+2000 elo
Stronger than Vice
Stronger than Zevra
Stronger than ISA
Stronger than Rubichess
Stronger than EveAnn
+2500 elo
Stronger than Schooner
Stronger than Counter
Stronger than Fruit 2.1
+2750 elo
Stronger than The Baron
Stronger than Tucano
Stronger than Fruit reloaded
Stronger than Gaviota
+3000 elo
Stronger than Hakkapeliitta 
Stronger than Hiarcs
Stronger than Pedone
Stronger than Vajolet
Stronger than Arasan
Stronger than Senpai
Stronger than Ethereal
Stronger than Laser
Stronger than Texel
Stronger than Rybka
Stronger than Critter
+3200 elo
Stronger than Gull
Stronger than Fritz
Stronger than Andscacs
Stronger than Fizbo
+3300 elo
Stronger than Shredder
Stronger than Fire
+3400
Stronger than Komodo
Stronger than Houdini
+3500
Stronger than Stockfish
King of the world
Error323 commented 6 years ago

I think we should strive for stockfish L8. And I think we can really do it. It'll depend on the community and compute we throw at it.

But bare in mind: 1 chess is much easier then go. 2 We're creating an engine optimized for chess, NNCache etc helps simulation times a lot. 3. I think we can get away with a much smaller network than 256x20. Their goal was a general purpose algorithm, we just want to be the best chess engine in the world :)

Zeta36 commented 6 years ago

I think the idea of @CMCanavessi is good. To have inside the readme.md a list of the achievements reached (and maybe some examples of games in a gif image) can be very positive in order to catch the attention. It would also be interesting to have a domain (instead of a IP) and a public server (amazon awz for example) where to install a wordpress with some good design and information about the project, the client download link, etc. We could even in the future try to execute the Tensorflow trained model in Javascript in the browser (using something like this: https://deeplearnjs.org/docs/api/index.html and with support for WebGL).

If we all donate a couple of euros or dollars (or bitcoins :D) we can rent that server for a year.

jkiliani commented 6 years ago

I have my doubts that a network smaller than 256x20 will suffice for Stockfish 8. At this point we're unfortunately only guessing at what network Deepmind used for AlphaZero since they never say, but 256x20 seems a good guess because of the AGZ paper.

They also noticed that with a large enough net, MCTS+NN scales better than Alpha Beta search, so they coincidentally evaluated at a huge amount of compute (4 TPU at 1 min/move). Aiming to beat Stockfish at shorter time controls and a less resource intensive setup is actually harder. A 256x20 net might do it, but if it didn't, the solution would be "Stack more layers".

However, some specific optimisations may help a lot. Implementing rotations may help the learning curve because of the training data enhancement, and once the policy priors are good enough, FPU reduction (https://github.com/gcp/leela-zero/commit/6b56e3ffbeb513e643e144397a4421e53c1879e5) may give the search a significant boost, although it probably wouldn't help yet right now.

Uriopass commented 6 years ago

I can buy a domain to redirect for like 2 years, It's not very expensive. What do you think of lczero.org ? Or maybe lczero.com ?

mooskagh commented 6 years ago

lczero.com is taken, and lczero.org I just happened to buy 30 minutes ago. Sorry, it seems the same idea came to use at the same time. :)

I'm happy to transfer it to whoever actually hosts 162.217.248.187 server, or to any other lczero developer.

On Fri, Mar 16, 2018 at 12:30 PM Douady Pâris notifications@github.com wrote:

I can buy a domain to redirect for like 2 years, It's not very expensive. What do you think of lczero.org ? Or maybe lczero.com ?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/glinscott/leela-chess/issues/120#issuecomment-373686603, or mute the thread https://github.com/notifications/unsubscribe-auth/AKvpl28J4eRJvhvKfrkUAeKmA24xBKqqks5te6JdgaJpZM4StELB .

Uriopass commented 6 years ago

No problem :-) I think @Error323 hosts the servers ? Or is it @glinscott ?

Zeta36 commented 6 years ago

Great!! we've got the domain. Now we would need install in a web server a nice wordpress with info of the project, links to download the client, beautiful charts, etc. Is there any designer in the room? :D

jkiliani commented 6 years ago

The server is @glinscott's, although @Error323 runs the training. Very good idea by the way to get a catchy domain name.

Are there any Chinese people here who can tell us what lczero.com is all about?

Error323 commented 6 years ago

Awesome people! @glinscott hosts the server! He did gave me full access rights, as such I'm synchronizing all the data across the world on my NAS so that I can train from home :smile: whilst also making a backup. It's best to coordinate with @glinscott. For now you can probably just point lczero.org to 162.217.248.187, sweet!

Regarding the engine achievements, I think it would be nice to annotate our ELO graph with markers where it beats a stockfish level. The ELO graph is now relative from random play against itself. These markers would give it a more sensible frame of reference for players.

CMCanavessi commented 6 years ago

The problem with annotating the ELO graphs comparing to SF levels, is that SF levels have not been tested by any serious rating list. We can deduce a bit what the aprox. strenght is, but I've only seen such numbers with SF playing against itself at different levels, kinda useless because, again, it's self-play.

mooskagh commented 6 years ago

I made it point to 162.217.248.187, http://lczero.org/ should work now. I've also shared that domain via google domains with both of you (you should get some email about it, I guess).

On Fri, Mar 16, 2018 at 1:05 PM F. Huizinga notifications@github.com wrote:

Awesome people! @glinscott https://github.com/glinscott hosts the server! He did gave me full access rights, as such I'm synchronizing all the data across the world on my NAS so that I can train from home 😄 whilst also making a backup. It's best to coordinate with @glinscott https://github.com/glinscott. For now you can probably just point lczero.org to 162.217.248.187, sweet!

Regarding the engine achievements, I think it would be nice to annotate our ELO graph with markers where it beats a stockfish level. The ELO graph is now relative from random play against itself. These markers would give it a more sensible frame of reference for players.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/glinscott/leela-chess/issues/120#issuecomment-373693646, or mute the thread https://github.com/notifications/unsubscribe-auth/AKvpl-PNmzp3BmTfrGf2G0AbW-lifOwUks5te6pzgaJpZM4StELB .

Error323 commented 6 years ago

Ah @CMCanavessi that makes sense. in that case we could annotate it with notable engines it wins from or is on par with?

edit: I fully agree we need solid elo numbers!

jkiliani commented 6 years ago

I came to the same conclusion as @CMCanavessi... it would be nice if Stockfish had settings with known, calibrated strength, but if so I don't know about it. At low skill levels (like 0), the scaling with time is almost negligible, at Level 10 it does scale a bit at least, but far less than the unhandicapped Stockfish does. And I can't really assign a reliable Elo rating to any of the settings, since I just have hearsay from the internet (Some people say e.g. Level 5 is 1400 Elo, others claim 1700).

I think Level 0 is probably around 900, but can't guarantee the accuracy.

luigio commented 6 years ago

One thing that would be nice to catch the attention of casual onlookers is a few sample games that showcase the latest network at full strength, since the training games are misleading in that respect.

CMCanavessi commented 6 years ago

I'm playing some matches now with a 40/40 time control (LCZero with 35.000 playouts move in 45-50 seconds on my PC) and I'm seeing it get draws vs engines in the 1500-1700 elo range. I will post later, 40/40 games are long. Maybe I should test some at 40/4?

luigio commented 6 years ago

Also crystal-clear instructions on the main page on how to contribute, of course. (Right now lczero.org has no such thing at all.)

Error323 commented 6 years ago

When #119 is approved and merged, this will be highest priority for me.

On Fri, Mar 16, 2018 at 4:57 PM, luigio notifications@github.com wrote:

Also crystal-clear instructions on the main page on how to contribute, of course.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/glinscott/leela-chess/issues/120#issuecomment-373758903, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEpOQK49dlOu76YdlVxok2j-JV6Vf9yks5te-DagaJpZM4StELB .

--

Kind regards,

Folkert Huizinga

mooskagh commented 6 years ago

There are lots of nice stats and graphs posted in groups (openings stats, how frequently every piece is moved, ratings against other engines, etc). It would be nice to add all of those (and more) to the site.

Also I guess it would be useful if all played games would be available for download, then more people would generate more stats.

On Fri, Mar 16, 2018 at 4:52 PM luigio notifications@github.com wrote:

One thing that would be nice for onlookers is a few sample games that showcase the latest network at full strength, since the training games are misleading in that respect.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/glinscott/leela-chess/issues/120#issuecomment-373757208, or mute the thread https://github.com/notifications/unsubscribe-auth/AKvplxB-SdfZiNHCTYkYigoR-GBKnb9nks5te9-0gaJpZM4StELB .

jjoshua2 commented 6 years ago

I asked someone on tcec chat to make a logo with leela stockfish hybrid and he made this for our new logo. Based on those two images.

mooskagh commented 6 years ago

I'd prefer something more minimalistic. :) Like, 1000 times more minimalistic.

On Fri, Mar 16, 2018 at 9:32 PM jjoshua2 notifications@github.com wrote:

I asked someone on tcec chat to make a logo with leela stockfish hybrid and he made this https://i.imgur.com/wCDrtCE.jpg for our new logo. Based on those two images.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/glinscott/leela-chess/issues/120#issuecomment-373836727, or mute the thread https://github.com/notifications/unsubscribe-auth/AKvpl4RsJEYEhKtkcoAb0m4ctNrTGwhrks5tfCFlgaJpZM4StELB .

CMCanavessi commented 6 years ago

Lol, that's a nice one, but we need a small one to use in GUIs like Arena, like the ones you can grab from http://www.chessmosaic.com/UCIEngines1.html

luigio commented 6 years ago

It's extremely ugly. :)

mooskagh commented 6 years ago

Also, the name (and domain name) is wrong. :)

On Fri, Mar 16, 2018 at 9:42 PM luigio notifications@github.com wrote:

It's extremelly ugly. :)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/glinscott/leela-chess/issues/120#issuecomment-373839134, or mute the thread https://github.com/notifications/unsubscribe-auth/AKvplwd9nvzO0bNHMqiAhdFK5D9fVs55ks5tfCOYgaJpZM4StELB .

glinscott commented 6 years ago

Wow, thank you!! That's really great.

On Fri, Mar 16, 2018 at 5:10 AM, mooskagh notifications@github.com wrote:

I made it point to 162.217.248.187, http://lczero.org/ should work now. I've also shared that domain via google domains with both of you (you should get some email about it, I guess).

On Fri, Mar 16, 2018 at 1:05 PM F. Huizinga notifications@github.com wrote:

Awesome people! @glinscott https://github.com/glinscott hosts the server! He did gave me full access rights, as such I'm synchronizing all the data across the world on my NAS so that I can train from home 😄 whilst also making a backup. It's best to coordinate with @glinscott https://github.com/glinscott. For now you can probably just point lczero.org to 162.217.248.187, sweet!

Regarding the engine achievements, I think it would be nice to annotate our ELO graph with markers where it beats a stockfish level. The ELO graph is now relative from random play against itself. These markers would give it a more sensible frame of reference for players.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/glinscott/leela-chess/issues/120# issuecomment-373693646, or mute the thread https://github.com/notifications/unsubscribe-auth/AKvpl- PNmzp3BmTfrGf2G0AbW-lifOwUks5te6pzgaJpZM4StELB .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/glinscott/leela-chess/issues/120#issuecomment-373694869, or mute the thread https://github.com/notifications/unsubscribe-auth/AAh3tMFbYAQoHqDmtSUPNpkWNz44fvKuks5te6vSgaJpZM4StELB .

CMCanavessi commented 6 years ago

Hi guys, I made a .7z package with all the needed files inside and a couple of .bat files so the end user only has to click them or use them in the GUI without any hassle.

Here's the package: http://www.mediafire.com/file/21k29oddu20lpz3/LCZero.7z

I'm also using lczero.exe with time control support, so it will always play at full strenght according to the time given by the GUI. This is good for testers and complaining users that think it's still too weak :)

I've included generation 9 network.

Ipmanchess commented 6 years ago

Thank you very much.. will check it out!

luigio commented 6 years ago

I can't make it work. It says it's not compatible with my Windows version (Windows 7, 32 bits, I have no GPU).

CMCanavessi commented 6 years ago

I have no GPU

There's your problem. It needs a GPU. There was also a CPU-only version, but it's been known to be broken for a couple of days. Don't know if that's still true, but in any case it would be very very slow.

Error323 commented 6 years ago

CPU version should be fixed.

On Sat, Mar 17, 2018 at 2:58 PM, Carlos M. Canavessi < notifications@github.com> wrote:

I have no GPU

There's your problem. It needs a GPU. There was also a CPU-only version, but it's been known to be broken for a couple of days. Don't know if that's still true, but in any case it would be very very slow.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/glinscott/leela-chess/issues/120#issuecomment-373922123, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEpORGXpNKEPpTnZTvulrYAI0zHhaf8ks5tfRaTgaJpZM4StELB .

--

Kind regards,

Folkert Huizinga

luigio commented 6 years ago

Still nothing. I downloaded client.exe from https://github.com/glinscott/leela-chess/releases

Is there supposed to be a different version somewhere? Instructions unclear. :(

CMCanavessi commented 6 years ago

If cpu-only is fixed, then you should download the binary from here: https://ci.appveyor.com/project/glinscott/leela-chess/build/job/lfhah1re5bo4tjbb/artifacts

Try that and comment if it worked :)

jjoshua2 commented 6 years ago

Fixed logo is here... https://i.imgur.com/3gcdf03.jpg

luigio commented 6 years ago

If cpu-only is fixed, then you should download the binary from here: https://ci.appveyor.com/project/glinscott/leela-chess/build/job/lfhah1re5bo4tjbb/artifacts

Try that and comment if it worked :)

It didn't work. I put it in the same folder where I had uncompressed the mediafire package from CMCanavessi. Then I tried running all .bat and .exe files there, to no avail.

jjoshua2 commented 6 years ago

Here is an alternate logo https://i.imgur.com/4kQl9ti.jpg it's simpler

mooskagh commented 6 years ago

I'm thinking more about something among those https://imgur.com/a/i6dsu lines. But that's probably on the other extreme of minimalism, and lacks some "nice touch".

On Sat, Mar 17, 2018 at 7:24 PM jjoshua2 notifications@github.com wrote:

Here is an alternate logo https://i.imgur.com/4kQl9ti.jpg it's simpler

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/glinscott/leela-chess/issues/120#issuecomment-373941821, or mute the thread https://github.com/notifications/unsubscribe-auth/AKvplzDjGkg-vqdktIpKhYKpHXeFE_6vks5tfVTEgaJpZM4StELB .

Uriopass commented 6 years ago

Since it's lc ZERO, makes sense that the logo is very minimal.

mooskagh commented 6 years ago

https://imgur.com/a/zJUxR

On Sat, Mar 17, 2018 at 8:42 PM Douady Pâris notifications@github.com wrote:

Since it's lc ZERO, makes sense that the logo is very minimal.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/glinscott/leela-chess/issues/120#issuecomment-373947319, or mute the thread https://github.com/notifications/unsubscribe-auth/AKvpl6aDToMxJzyEc-KTS9q_OfLMywHAks5tfWc5gaJpZM4StELB .

CMCanavessi commented 6 years ago

Love them both!! But I just selected the upper one for my tournaments, may I use it?

mooskagh commented 6 years ago

Sure! Feel free to use any of them.

mooskagh commented 6 years ago

https://imgur.com/a/qQFut another variation of the same idea.

CMCanavessi commented 6 years ago

They all look cool and have that "retro" feel that i love, but I still like the more minimalist L c 0 that you did yesterday. That's the best for me and what I'll use :D

zz4032 commented 6 years ago

Only thing the minimalistic version is missing is some relation to chess.

mooskagh commented 6 years ago

Yes, it depends on what it will be used for.

If it is used at chess engine tournaments, in that context I'd actually avoid having anything chess related in logo. Here http://www.chessmosaic.com/ListOfAllEngines1.html everyone tries to squeeze something chess related in their logo, and it seems that it's not really necessary and it seems that something minimalistic would look better.

In isolation from places with lots of chess-related logos, I agree that it would be nice to have some chessy touch in logo. If trying to keep that 7-segment LED display idea, we can try to add some chess clock elements (https://files.chesscomfiles.com/images_users/tiny_mce/AnthonyCG/GardeDig15.JPG for inspiration), although I don't know how recognizable it would be.