computer-chess-championship

[l1t1]

https://www.chess.com/computer-chess-championship leela chess zero now at 5th

[Friday9i]

Yeah, that's nice as it is already sure to be selected for the second round as one of the 8 best engines! And it's a quite an old version of LC0 (ie an old network, quite far from the best) Unfortunately, it's not possible to update for the second round, so it has very little chances of beating current top engines (Stockfish, Komodo and Houdini). But for next tournament in a few months, it may win...

[l1t1]

https://github.com/LeelaChessZero/lc0/releases

https://github.com/LeelaChessZero/lc0/wiki/Getting-Started We are transitioning to a completely rewritten engine (lc0.exe), from the original (lczero.exe). The main reason is for speed, up to 10X faster for Nvidia GPUs. did somene test leelazero on Nvidia cuda?

[john45678]

For a review of some games played by Leela Chess at CCCC see Kingscrusher yt: https://www.youtube.com/user/kingscrusher

[l1t1]

it seeems publish software include cuda dll is permitted

Release lc0-windows-cuda.zip package now contains NVdia CUDA and cuDNN .dlls.

[ghost]

This is an achievement, however when comparing the ML engines to A/B engines, the hardware isn't equal for this event.

CPU Engines: CPUs: 2 x Intel Xeon Platinum 8168 @ 2.70 GHz 33 MB L3 Cores: 48 physical (96 logical) RAM: 256GB DDR4-2666 ECC Registered RDIMM SSD: 2x Crucial MX300 (1TB) in RAID1 OS: Windows Server 2016

GPU Engines: GPU: 4x Tesla V100 (64 GB GPU memory) CPU: Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30 GHz Cores: 16 physical (32 virtual) RAM: 256 GB

[l1t1]

@thalessa which equipment did lc0 use in the championship?

[jkiliani]

@thalessa The hardware used may not be equal in purchasing price, but:

• The GPU server is rented, and obviously determined by availability of preconfigured GPU servers on the cloud, with sufficient stability that they could be sure the TCEC debacle wouldn't be repeated
• Even though V100 are very expensive to buy, the newly released RTX 2080Ti provides only very slightly less performance for a fraction of the price
• Lc0 doesn't currently scale all that well with multiple GPUs. The improvement from 2x V100 to 4x V100 is actually marginal

So the argument that Leela Chess Zero had an unfair hardware advantage actually falls flat on closer inspection. Just because those GPU servers were used doesn't mean that a much cheaper setup with almost equal performance wasn't available, or at least won't be available in the very near future.

[tapsika]

I don't think hardware prices has any significance here, only performance (gflops/mips). You can even compare the number of cores (orders of magnitude difference) to get a very rough idea.

Btw I'm not familiar with recent chess development, have someone tried parallelizing a traditional engine over GPUs? I suppose it would be less effective during A/B search, but maybe something can be done with batching?

[l1t1]

I got it http://blog.lczero.org/2018/08/cccc-starts.html Leela will play on four Tesla V100 GPUs while the other engines on 46 threads of a 2 x Intel Xeon Platinum 8168, 2.70 GHz that has 48 logical cores and 96 threads.

[gcp]

it seeems publish software include cuda dll is permitted

The situation is massively more complicated than this. a) lc0 has an NVIDIA engineer contributing, and he apparently got permission for bundling the DLLs b) the lc0 client was re-licensed to include a license exemption for linking with cuDNN.

[gcp]

Leela Chess Zero has its own github, this is totally offtopic here.

[ghost]

@thalessa So the argument that Leela Chess Zero had an unfair hardware advantage actually falls flat on closer inspection. Just because those GPU servers were used doesn't mean that a much cheaper setup with almost equal performance wasn't available, or at least won't be available in the very near future.

Why did you assume I thought it was unbalanced towards Leela?

I think the comparison is invalid.

[l1t1]

lc0 now at 4th

[l1t1]

[ghost]

The random Leela loss to the weak engine Andscacs. I guess that's their equivalent of "10 dan that can't read ladders"?

[Matuiss2]

andscacs is not weak at all

[l1t1]

stockfish is INVINCIBLE Houdini & Komodo both win more games than lc0, but also lost more games.

[l1t1]

lc0 lost to fire, the 6th program

[ghost]

andscacs is not weak at all

It's at the bottom of the table. Compartively speaking, it is.

[l1t1]

lc0 lost two games to andscacs

[l1t1]

I wonder why the opensource stockfish is stronger than some closesource program? Is the weight file published?

[ghost]

I wonder why the opensource stockfish is stronger than some closesource program? Is the weight file published?

The chess programs - except for leela etc - do not use weights as they are not machine learning. They calculate an evaluation for the board using conventional, hand-written algorithms.

Stockfish is open source and has an extensive distributed testing network called fishtest (http://tests.stockfishchess.org/tests) where volunteers can donate computers and try out modification to the algorithms.

The main rivals Houdini and Komodo are commercial so the developers pay for the test hardware themselves, and only they have the code to try changes. They don't have the computing power of stockfish, and fewer people trying out code changes. However they have beaten Stockfish during some years as code and patches change.

Stockfish recently was called "drawfish" as it played always what it saw as the best move. This lead to a lot of draws that could have been wins against weaker engines, if Stockfish had played a move that it believed was worse but the other engine didn't understand because it was bad at chess. So Stockfish lost a tournament despite never losing a game!

Now they have implemented "contempt" that allows Stockfish to try to win by disrespecting the opponent, playing suboptimal moves that can beat weaker engines who can't understand them, but would lead to a loss in an equal skill (stockfish v stockfish) game.

[l1t1]

@thalessa thanks a lot

[l1t1]

lc0 is 4th now, also has more draw games than the 3rd

[l1t1]

finished

[TFiFiE]

The random Leela loss to the weak engine Andscacs. I guess that's their equivalent of "10 dan that can't read ladders"?

With respect to chess, the tactical blind spot in an otherwise superstrong zero engine seems to be discovered attacks, although LCZero has seemingly moved past that now.

[l1t1]

stockfish got a fail stockfish got a fail again

[l1t1]

why didn't black resign? is there win by resign in a match of chess ?

[roy7]

The bots aren't set up to resign, they always fight to the bitter end.

[ghost]

Resignation will be a tournament rule set by the tournament. Sometimes its when both engine evals are heavily in one side, sometimes it's by tablebase (a database of endgames).

However I think that in a game with neural net engines, the ending should be played, as they sometimes miss tactics (the ladder problem). Leela Chess Zero famously blundered the Lucena position when it played against IM Lovlas. It's a very standard well-studied position and any top classical engine would not have misplayed it.

https://en.wikipedia.org/wiki/Lucena_position

https://lichess.org/study/kLXrkT4M

[l1t1]

lc0 3:0 leading

[l1t1]

lc0 seems too optimistic to evaluate the position

[l1t1]

komodo underestimate

[l1t1]

lc0 win

[l1t1]

lc0 beat stockfish

[l1t1]

lc0 lost too much games this time

[l1t1]

lc0 got wrost result

[l1t1]

lc0 18.1

[l1t1]

LCZero blog

2018-10-19

Leela beats Fire promoting to Semi-Final of TCEC Cup!

Leela in a classic drama style, promoted in TCEC Cup Semi-Finals and it will face Stockfish today! While in CCCC blitz tournament she is still at 3rd place ahead of Komodo, Ethereal and Fire and behind Stockfish and Houdini.

Read more »