Benchmarks?

[ghost]

I'm wondering what ms/move other people are getting with their hardware. I'll start: GTX 1060 3GB + i5-2500K (stock) (Leela Zero 0.9 + AutoGTP v8, 5x64) -g 2: 1270 ms/move -g 3: 1230 ms/move

-g 2 produces significantly less heat so it's probably the option I'll stick with.

[txinbo]

GTX 1080 + i7-7700 (stock) win10 4 instances 1 thread, 2885-3004ms/move (Leela Zero 0,9 + autogpt v8)

[Dorus]

i7-2600k, GTX 760 1 instances, 2 game mode. Default settings (= 1 thread?) 2650ms/move.

I have tested 1 instance before on version 0.6 and it was generating 20% less games per hour than 2 instances, so i settled for 2.

[zediir]

i7-7700k + 2xGTX 1080 Win10. Four instances running with two per GPU. 532ms/move. (Leela Zero 0,9 + autogpt v8)

Now running a test with autogtp -g3 -u0 -u1 so 6 instances. Seems to be faster. 438ms/move

[SharlatanY]

Ryzen 1700x (Stock) + 1080Ti on Windows with Leela Zero 0.9 + AutoGTPv8, 5x64 Optimum for me seems to be 4 instances which result in an average of 3169ms/move per instance or about 792ms/move on average, if you will.

But you might get better results with the same setup in a Linux environment, as discussed here: https://github.com/gcp/leela-zero/issues/295

[BHydden]

Windows 10, GTX 760 2GB, i5 750 2.66GHz Leela zero 0.8 + autogtpv7 5x64 3 instances running 1 game each 7453-7489-7493 ms/move respectively

[barrtgt]

Win10, Leela Zero 0.10.1 i7-7700K, RX 480 Driver 17.12.2

workload: graphics gpu -g 1: 1600ms/m 10% cpu gpu -g 2: unstable cpu -g 1: 6000ms/m 14% cpu cpu -g 2: 3000ms/m 25% cpu cpu -g 4: 1775ms/m 49% cpu cpu -g 6: 1650ms/m 73% cpu cpu -g 8: 2075ms/m 98% cpu

workload: compute (before restart) gpu -g 1: 1150ms/m 10% cpu

gpu -g 1,cpu -g 2: 1200ms/m,3250ms/m 35% cpu combined: 876ms/m gpu -g 1,cpu -g 4: 1400ms/m,2050ms/m 60% cpu combined: 832ms/m gpu -g 1,cpu -g 6: 1800ms/m,1950ms/m 85% cpu combined: 936ms/m

workload: compute (after restart) gpu -g 1: 1600ms/m 9% cpu

If a restart is made, cycling from compute to graphics and back will restore the previous speeds.

[sFaurite]

Win10 VS Ubuntu 17.10 Core i5 7600K @ 4.5GHz GTX 1070 8GB

Version			Network
Leelaz	0.9		hash	1607dbea
Autogtp	0.8		5 blocks	64 filters
Linux
nb autogtp	mean ms/move	total ms/move	%GPU	%CPU
1	1031	1031	61	25
2	1393	696	100	50
3	2097	699	100	75
4	3394	849	80	100
Windows
nb autogtp	mean ms/move	total ms/move	%GPU	%CPU
1	2139	2139	37	25
2	2140	1070	45-74	50
3	2668	889	86	75
4	3385	846	86	100
5	4224	845	87	100
Best Linux	696	18%	faster
Best Windows	845

200W measured at the outlet : that's around 0,6€ per day. And since it is winter now it is essentially free.

[aappleby]

Hello from Reddit (https://www.reddit.com/r/nvidia/comments/7iq5tk/just_got_a_titan_v_this_morning_ask_me_to/)

Thought I'd post this here - my numbers aren't as good as some of y'all are seeing, not sure why.

Leela Zero 0.9 + AutoGTP v8, Win10, I7-8700k, Titan V at stock everything: autogtp.exe -g4 4 game(s) played in 22 minutes = 342 seconds/game, 923 ms/move, last game took 1369 seconds.

[bastiaf]

Leela Zero 0.9 on Windows 10 i7-7700K MSI GTX 1080 GAMING X 8G (but not overclocked)

1 instance: 2 game(s) played in 21 minutes = 646 seconds/game, 2045 ms/move, last game took 643 seconds.

2 instances: 72 (M6) 47 game(s) played in 280 minutes = 358 seconds/game, 944 ms/move, last game took 1325 seconds.

3 instances: 13 game(s) played in 57 minutes = 263 seconds/game, 733 ms/move, last game took 17 seconds. (Oh, a 17 seconds game. I didn't see that before.)

4 instances: 39 game(s) played in 167 minutes = 257 seconds/game, 744 ms/move, last game took 1125 seconds.

I use mostly 2 instances with nice level 19, so the PC can be still used for everything else. GPU usage is at 50% than, CPU only 27%. And yes, it is used for heating too. ;)

[NelsonMinar]

i7-7700K, GTX 1080 9% overclocked, Windows 4 instances: 286 seconds/game, 701 ms / move

1 vCPU (Google Compute), K80 GPU, Linux. 4 instances; 1113 seconds/game, 2100 ms / move.

[navghtivs]

With v0.9:

i7-4770K, GTX-780, Windows 10: -g3: 1720ms/move

And I got some aws credit from last Month and used some of it yesterday to spin up an aws p3.2xlarge instance for a couple of hours but didn't have enough time to figure out best configuration: Linux(Ubuntu) with 1 NVIDIA Tesla V100, tried both -g4 and -g10: 550ms/move. The GPU used 88W out of maximum 300W so I feel there are ways to push it harder.

[MartinVingerhoets]

Ryzen 1600X Cpu only Linux with -g 1(12 threads):
~15-20 minutes/game 2500 ms/move 3-4 games/hour

[marcocalignano]

Sweet, Xmas is early this year! My new GeForce 1050 ti just arrived. I will try it this evening!

[marcocalignano]

Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz Linux (Kubuntu 17.10)

GTX 1050 ti : 24 game(s) (24 self played and 0 matches) played in 218 minutes = 545 seconds/game, 1333 ms/move, last game took 2921 seconds.

GTX 760 10 game(s) (10 self played and 0 matches) played in 207 minutes = 1245 seconds/game, 2912 ms/move, last game took 2305 seconds.

And they are running at the same time probably if I let the CPU all for the 1050 I get more out of it!

[zediir]

With i7-7700k + 2xGTX 1080 Win10 and autogtp -g4 -u0 -u1

81 seconds/game, 275 ms/move with the latest /next so almost twice as fast as when I last posted

[marcocalignano]

try autogtp -g6 -u0 -u1

[zediir]

That would start 12 leelaz instances on selfplay (-g parameter is per gpu). But I tried it anyway.

12 game(s) (12 self played and 0 matches) played in 23 minutes = 119 seconds/game, 254 ms/move

Also ran -g4 -u0 -u1 again so there were no match games in the ms/move calculations

6 game(s) (6 self played and 0 matches) played in 14 minutes = 140 seconds/game, 258 ms/move

(had to stop this one before 8 games finished so the seconds/game figure is not accurate)

So no significant difference.

[marcocalignano]

well is 5 minutes less for the same number of games and I guess you could try even autogtp -g8 -u0 -u1

[zediir]

Note that I still had 2 of the first started games going and only 12 or 6 games does not give accurate games/minute figure because the length of games varies.

Only 1.5% difference in the ms/move values

[marcocalignano]

actually you have 12 games going in the first case and 8 games going in the second case "at all times"

[zediir]

True but the ms/move is an average of the move speeds of all the 12 going on games or 8 going on games so not much difference. No way it gave that big a performance increase when gpu usage was already almost full with -g4

Here's the -g6 when 6 games had been played

6 game(s) (6 self played and 0 matches) played in 14 minutes = 148 seconds/game, 248 ms/move

-g8 would also take around 10 gigs of memory when doing matches.

[marcocalignano]

Before the last version I had 530 seconds/game, 910 ms/move now I get 216 seconds/game, 618 ms/move. Is that possible?

[jkiliani]

The current result sounds correct, the one with the last version suspicious, because it would implicate a game lasts more than 600 moves. There must have been a calculation error that was fixed, I think I remember a commit like that but don't know when it was pushed exactly.

[barrtgt]

So with the option to use a CPU OpenCL version, I wanted to combine the rates of the two autogtp instances and I found this: (a * b)/(a + b)

[zediir]

3046 ms/move on a 2012 Mac mini with Intel HD Graphics 4000 using the winograd branch.

[roy7]

For my Ryzen 1700 + 1080Ti:

Current /next:

9 game(s) (9 self played and 0 matches) played in 30 minutes = 201 seconds/game, 622 ms/move, last game took 299 seconds.

Current /winograd:

35 game(s) (34 self played and 1 matches) played in 110 minutes = 189 seconds/game, 571 ms/move, last game took 229 seconds.

Edit: I forgot to try -g. Oops.

[zediir]

2xGTX1080 winograd: 202 ms/move. I'm cpu bound again. Booo. Squeezed some more performance by overclocking from 4.2 GHz to 4.58 GHz

[tux3]

I have a 1060 6GB + 1700X (slightly overclocked, fast RAM) running with -g 2 on Linux next: 570 ms/move winograd: 350 ms/move

Winograd is a pretty huge improvement for me! Especially given that I don't have the highest-end GPU.

[sFaurite]

gtx 1070 on linux with -g3

next : 1236 game(s) (1181 self played and 55 matches) played in 2949 minutes = 143 seconds/game, 454 ms/move, last game took 702 seconds.

GPU usage is 100%

winograd : 17 game(s) (17 self played and 0 matches) played in 26 minutes = 92 seconds/game, 255 ms/move, last game took 206 seconds.

But now GPU usage is slightly under 100% so I tried with -g 4 but CPU usage is 100% and GPU get worse, so I am CPU bounded now :-(

But with bigger network size it will change back to GPU bounded so it is not that bad.

I am still using next since winograd is not yet approved by @gcp