search

leela-zero / leela-zero

Go engine with no human-provided knowledge, modeled after the AlphaGo Zero paper.
GNU General Public License v3.0
5.36k stars 1.01k forks source link

CGOS timed out? #990

Closed jjoshua2 closed 6 years ago

jjoshua2 commented 6 years ago

CGOS is timing out, but my lz12 is still using some cpu and gpu, but not much, I guess to ponder since it never got genmove? (ponder is not disabled) Not sure what is wrong. EDIT I won on time not lost I think end of cogs logs:

23:04:28    C->E time_left w 876 0
23:04:28    E->C = 
23:04:28    C->E genmove w
23:04:36    E->C = D14
23:04:36    C->S D14
23:04:42    S->C play b j16 874312
23:04:42    C->E play b j16
23:04:42    E->C = 
23:04:42    S->C genmove w 867971
23:04:42    C->E time_left w 867 0
23:04:42    E->C = 
23:04:42    C->E genmove w
23:04:51    E->C = R3
23:04:51    C->S R3
23:05:12    S->C info Maximum time until next round: 28:41
23:05:12    Maximum time until next round: 28:41
23:06:27    S->C info Maximum time until next round: 27:36
23:06:27    Maximum time until next round: 27:36
23:07:42    S->C info Maximum time until next round: 26:30
23:07:42    Maximum time until next round: 26:30

end of engine logs:

Thinking at most 12.5 seconds...
NN eval=0.524154
Playouts: 5183, Win: 51.91%, PV: D16 R16 D4 P16 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 R15 Q15 R14
Playouts: 9782, Win: 52.12%, PV: D16 D3 Q16 D5 R3 R4 Q3 P3 P2 O2 O3 P4 N2 Q2 O1 R2 E3 E4 F3 D2 F4 R17
Playouts: 14156, Win: 52.27%, PV: D16 D3 Q16 D5 R3 R4 Q3 P3 P2 O2 O3 P4 N2 Q2 O1 R2 R14 C17 C16 D17 E17 E18 F18 F17 E16 G18
Playouts: 18379, Win: 52.35%, PV: C16 D3 Q16 R17 R16 Q17 P17 P18 O18 O17 P16 N18 Q18 O19 R18 E17 D14 D6 R3 R4 Q3 P3 P2 O2 O3 P4 N2 Q2 O1 R2 C8 C12 D4 E4 C4 C3
0.8s left
Stopping early.

 C16 ->    5702 (V: 52.83%) (N:  7.69%) PV: C16 R16 D4 E17 D14 O16 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 C6 H17 D17
 D16 ->    4318 (V: 52.36%) (N: 22.45%) PV: D16 D3 Q16 D5 R3 R4 Q3 P3 P2 O2 O3 P4 N2 Q2 O1 R2 R14 C17 C16 D17 E17 E18 F18 F17 E16 G18 D18 F19 C18
 D17 ->    3664 (V: 52.73%) (N:  7.42%) PV: D17 D3 Q16 C15 F16 D5 R3 R4 Q3 P3 P2 O2 O3 P4 N2 Q2 O1 R2 O17 C12
  C4 ->    2240 (V: 52.57%) (N:  7.42%) PV: C4 R16 D16 E3 D6 C17 D17 C16 C15 B15 B14 C14 D15 B13 B16 A14 B17 O16 R3 Q3
  D4 ->    1318 (V: 51.67%) (N: 14.55%) PV: D4 D16 Q16 C3 D3 C4 C5 B5 B6 C6 D5 B7 B4 A6 B3 R17 Q17 R16 R15 S15 S14 R14 Q15 S13 S16 T14 S17 Q6 C17 C16
 Q16 ->    1316 (V: 51.59%) (N: 15.49%) PV: Q16 D16 D4 C3 C4 D3 E3 E2 F2 F3 E4 G2 D2 F1 C2 R17 R16 Q17 P17 P18 O18 O17 P16 N18 Q18 O19 R18 C14 R3 Q3
 Q17 ->     920 (V: 52.08%) (N:  6.91%) PV: Q17 D16 D4 R15 O16 C3 C4 D3 E3 E2 F2 F3 E4 G2 D2 F1 C2 R12 C17 D17
  D3 ->     886 (V: 52.05%) (N:  6.87%) PV: D3 R16 D16 C17 D17 C16 C15 B15 B14 C14 D15 B13 B16 A14 B17 C5 F4 O16 R3 Q3 R4
 R16 ->     789 (V: 51.97%) (N:  6.67%) PV: R16 D3 D16 D5 R3 R4 Q3 P3 P2 O2 O3 P4 N2 Q2 O1 R2 O16
  R3 ->      69 (V: 51.41%) (N:  0.93%) PV: R3 R4 Q3 P3 P2 O2 O3 P4 N2 Q2 O1 R2 D3 R16 D16 C5
21223 visits, 7430520 nodes, 21222 playouts, 1809 n/s

= C16

 R16 ->    3063 (V: 47.08%) (N: 21.02%) PV: R16 D4 E17 D14 O16 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 F3 H17 C9 R10 Q6 P10
  D3 ->    3051 (V: 47.01%) (N: 23.52%) PV: D3 Q16 R17 R16 Q17 P17 P18 O18 O17 P16 N18 Q18 O19 R18 E17 D14 D6 R3 R4 Q3 P3 P2 O2 O3 P4 N2 Q2 O1 R2 C8 C12 D4 E4 C4 C3 B3 F3 B5 C7
  C4 ->    2469 (V: 47.08%) (N: 16.96%) PV: C4 Q16 R17 R16 Q17 P17 P18 O18 O17 P16 N18 Q18 O19 R18 E17 D14 F4 R3 R4 Q3 P3 P2 O2 O3 P4 N2 Q2 O1 R2 G17 J17 G15 G18
 Q17 ->    2315 (V: 47.23%) (N: 12.14%) PV: Q17 D4 E17 D14 Q15 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 C6 J17 J3 C12
  D4 ->    1856 (V: 47.09%) (N: 13.16%) PV: D4 Q16 R17 R16 Q17 P17 P18 O18 O17 P16 N18 Q18 O19 R18 E17 D14 O3 C3 C4 D3 E3 E2 F2 F3 E4 G2 D2 F1 C2 G17 C17
 Q16 ->    1120 (V: 47.00%) (N:  8.64%) PV: Q16 D4 E17 D14 C3 D3 C4 C5 B5 B6 C6 D5 B7 B4 A6 B3 H17 R3 Q3 R4 R5 S5 S6 R6 Q5
 E17 ->      23 (V: 45.89%) (N:  0.45%) PV: E17 Q16 D15 C15 D14
 D16 ->       9 (V: 45.39%) (N:  0.23%) PV: D16 D17 D3 Q16 C15

13907 visits, 4839784 nodes

>>play b q16
= 

>>time_left w 888 0
= 

1 visits, 359 nodes

>>genmove w
Thinking at most 12.3 seconds...
NN eval=0.521789
Playouts: 4968, Win: 52.98%, PV: D4 E17 D14 C3 D3 C4 C5 B5 B6 C6 D5 B7 B4 A6 B3 H17 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 R17 Q17 R16 R15 S15 S14 R14 Q15 S13
Playouts: 9334, Win: 53.08%, PV: D4 E17 D14 C3 D3 C4 C5 B5 B6 C6 D5 B7 B4 A6 B3 H17 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 R17 Q17 R16 R15 S15 S14 R14 Q15 S13 S16 T14 S17 Q14 P7 K4
Playouts: 13600, Win: 53.16%, PV: D4 E17 D14 C3 D3 C4 C5 B5 B6 C6 D5 B7 B4 A6 B3 R6 R17 R16 Q17 P17 P18 O18 O17 P16 N18 Q18 O19 R18 G17 N15 L16 Q10 O3 P3 O4 L3 L4 K4
Playouts: 17735, Win: 53.17%, PV: D4 E17 D14 C3 D3 C4 C5 B5 B6 C6 D5 B7 B4 A6 B3 H17 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 R17 Q17 R16 R15 S15 S14 R14 Q15 S13 S16 T14 S17 Q14 P7 K4 R8 Q8
1.5s left
Stopping early.

  D4 ->    9680 (V: 53.10%) (N: 58.87%) PV: D4 E17 D14 C3 D3 C4 C5 B5 B6 C6 D5 B7 B4 A6 B3 H17 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 R17 Q17 R16 R15 S15 S14 R14 Q15 S13 S16 T14 S17 Q14 P7 K4 R8 Q8 Q7
  C4 ->    6983 (V: 53.43%) (N: 21.84%) PV: C4 E17 D14 E3 D6 H3 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 R17 Q17 R16 R15 S15 S14 R14 Q15 S13 S16 T14 S17 H17 P7 Q9 C8
  D3 ->    2170 (V: 53.10%) (N: 13.86%) PV: D3 E17 D14 C5 F4 C8 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 R17 Q17 R16 R15 S15 S14 R14 Q15 S13 S16 T14 S17 H17 C17 B17 D16
 R17 ->      90 (V: 51.99%) (N:  1.45%) PV: R17 Q17 R16 R15 S15 S14 R14 Q15 S13 S16 T14 S17 D4 E17 D14 C3 C4 D3
  R3 ->      39 (V: 50.26%) (N:  1.23%) PV: R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 D4 E17 D14
 E16 ->       9 (V: 47.65%) (N:  0.43%) PV: E16 D3 C5 F4
18972 visits, 6488998 nodes, 18971 playouts, 1758 n/s

= D4

 E17 ->   12217 (V: 47.00%) (N: 56.74%) PV: E17 D14 C3 D3 C4 C5 B5 B6 C6 D5 B7 B4 A6 B3 H17 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 R17 Q17 R16 R15 S15 S14 R14 Q15 S13 S16 T14 S17 Q14 P7 K4 R8 Q8 Q7
  C3 ->    4640 (V: 46.77%) (N: 29.91%) PV: C3 D3 C4 C5 B5 B6 C6 D5 B7 B4 A6 B3 E17 D14 R6 R17 R16 Q17 P17 P18 O18 O17 P16 N18 Q18 O19 R18 G17 N15 L16 Q10 O3 P3 O4 L3 L4 K4
  R6 ->     377 (V: 46.59%) (N:  2.99%) PV: R6 R17 Q17 R16 R15 S15 S14 R14 Q15 S13 S16 T14 S17 F16 C3 D3 C4 C5 B5 B6 C6 D5 B7 B4 A6 B3 H17 P2 D16 D15 D17
 R14 ->     139 (V: 46.67%) (N:  1.02%) PV: R14 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 F16 C3 D3 C4 C5 B5 B6 C6 D5 B7 B4 A6 B3
 D16 ->     138 (V: 45.77%) (N:  2.18%) PV: D16 D17 E17 D15 E16 C17 C3 D3 C4 C5 B5 B6 C6 D5 B7 B4 A6 B3 J17
 O17 ->      41 (V: 46.56%) (N:  0.35%) PV: O17 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 F16 C3
  O3 ->      31 (V: 45.81%) (N:  0.45%) PV: O3 R17 Q17 R16 R15 S15 S14 R14 Q15 S13 S16 T14 S17 F16 C3 D3 C4
  Q6 ->      12 (V: 45.46%) (N:  0.24%) PV: Q6 R17 R16 Q17 P17 P18 O18

17596 visits, 5924111 nodes

>>play b e17
= 

>>time_left w 876 0
= 

 D14 ->       2 (V: 49.49%) (N: 48.22%) PV: D14 C3
 D17 ->       1 (V: 44.24%) (N: 16.54%) PV: D17 

4 visits, 1424 nodes

>>genmove w
Thinking at most 12.1 seconds...
NN eval=0.590494
Playouts: 15123, Win: 52.98%, PV: D14 C3 D3 C4 C5 B5 B6 C6 D5 B7 B4 A6 B3 H17 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 R17 Q17 R16 R15 S15 S14 R14 Q15 S13 S16 T14 S17 Q14 P7 K3 R8 Q8 Q7 R9 Q9
Playouts: 18809, Win: 52.99%, PV: D14 C3 D3 C4 C5 B5 B6 C6 D5 B7 B4 A6 B3 R6 R17 R16 Q17 P17 P18 O18 O17 P16 N18 Q18 O19 R18 G17 N15 L16 Q10 O3 P3 O4 L3 L4 M4 M3
Playouts: 22796, Win: 52.95%, PV: D14 C3 D3 C4 C5 B5 B6 C6 D5 B7 B4 A6 B3 H17 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 R17 Q17 R16 R15 S15 S14 R14 Q15 S13 S16 T14 S17 Q14 P7 K4 R8 Q8 Q7 R9 Q9
4.1s left
Stopping early.

 D14 ->   16046 (V: 53.09%) (N: 48.22%) PV: D14 C3 D3 C4 C5 B5 B6 C6 D5 B7 B4 A6 B3 H17 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 R17 Q17 R16 R15 S15 S14 R14 Q15 S13 S16 T14 S17 Q14 P7 K4 R8 Q8 Q7 R9 Q9
  R3 ->    3961 (V: 53.00%) (N: 13.87%) PV: R3 D15 C15 D14 C13 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 R17 Q17 R16 R15 S15 S14 R14 Q15 S13 S16 T14 S17 D13 C3 C4 D3 E3 E2 F2 F3 E4 G2 D2 F1 C2 F4 H17 F14 L17
 D17 ->    1686 (V: 52.23%) (N: 16.54%) PV: D17 E16 D14 J17 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 R17 Q17 R16 R15 S15 S14 R14 Q15 S13 S16 T14 S17 F3 P13 Q11 C15 D15
 R17 ->    1519 (V: 52.66%) (N:  9.92%) PV: R17 Q17 R16 R15 S15 S14 R14 Q15 S13 S16 T14 S17 D14 C3 C4 D3 E3 E2 F2 F3 E4 G2 D2 F1 C2 H17 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4
 C13 ->     338 (V: 52.55%) (N:  2.43%) PV: C13 C3 C4 D3 E3 E2 F2 F3 E4 G2 D2 F1 C2 O17 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 H17
 D15 ->      37 (V: 50.69%) (N:  0.77%) PV: D15 H17 R3 Q3 R4 R5 S5 S6
 E16 ->      10 (V: 47.99%) (N:  0.44%) PV: E16 D17 C17 F16 E15
23598 visits, 7903630 nodes, 23597 playouts, 2931 n/s

= D14

  C3 ->   16749 (V: 46.99%) (N: 57.43%) PV: C3 D3 C4 C5 B5 B6 C6 D5 B7 B4 A6 B3 O3 R17 Q17 R16 R15 S15 S14 R14 Q15 S13 S16 T14 S17 H17 C17 B17 D16 C15 R10 Q14 R12 R6 R5 Q6
 H17 ->    3787 (V: 46.80%) (N: 18.11%) PV: H17 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 C6 C12 D17 E16 C13 D12 F14 C8 K17 O17 J15 K16 J16 J17
 R14 ->     700 (V: 46.48%) (N:  5.16%) PV: R14 R3 R4 Q3 P3 P2 O2 O3 P4 N2 Q2 O1 R2 H17 L17 F16 C17 B17 D16 C18
  R6 ->     685 (V: 46.68%) (N:  3.95%) PV: R6 R17 Q17 R16 R15 S15 S14 R14 Q15 S13 S16 T14 S17 H17 C3 D3 C4 C5 B5 B6 C6 D5 B7 B4 A6 B3 C17 B17 D16 C15 K17 K16
 O17 ->     586 (V: 46.64%) (N:  3.51%) PV: O17 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 C6 R10 H17 R8 Q6 P8
 C17 ->     150 (V: 46.48%) (N:  1.10%) PV: C17 B17 D16 C18 C15 D17 B16 C17 D15 E18 F17 R17 Q17 R16 R15 S15
 J17 ->     132 (V: 46.53%) (N:  0.88%) PV: J17 R3 Q3 R4 R5 S5 S6 R6 Q5 S7 S4 T6 S3 R17 Q17 R16 R15 S15 S14 R14 Q15
  O3 ->      69 (V: 46.09%) (N:  0.74%) PV: O3 R17 Q17 R16 R15 S15 S14 R14 Q15 S13 S16 T14 S17 F3 H17 R6
  C6 ->      25 (V: 46.31%) (N:  0.22%) PV: C6 C5 D6 F4 C10 R3 R4 Q3
  F3 ->      25 (V: 46.12%) (N:  0.25%) PV: F3 E3 F4 D6 K4 R17 Q17
 Q14 ->      21 (V: 44.67%) (N:  0.48%) PV: Q14 R3 R4 Q3 P3 P2 O2 O3 P4 N2 Q2 O1
  C4 ->      15 (V: 46.19%) (N:  0.16%) PV: C4 C3 C5 D3 C8 R3 Q3 R4
 O16 ->       9 (V: 45.39%) (N:  0.15%) PV: O16 R3 R4 Q3 P3
  Q6 ->       6 (V: 43.57%) (N:  0.18%) PV: Q6 R17 R16 Q17 P17

22960 visits, 7634716 nodes

>>play b j16
= 

>>time_left w 867 0
= 

1 visits, 355 nodes
jjoshua2 commented 6 years ago

Apparently I won by time it appears lol? Not sure why gpu usage was only 20% with 4 threads but its going up to 55% now. Maybe pondering for a long time causes gpu usage to drop over time?

killerducky commented 6 years ago

What's your LZ command line? From the logs I can see it is definitely pondering. What is your bot's CGOS name?

23:04:42    C->E genmove w
23:04:51    E->C = R3
23:04:51    C->S R3

The CGOS client asked for LZ to genmove w and LZ sent R3, and the client sent that move to the server. Then nothing happened. It looks like you actually won the game and it's the opponent who timed out. In your engine log the R3 move doesn't appear, it might just be a disk buffering issue.

If it sat there pondering while the game was actually already over it probably hit the node limit and then... Well it depends on exactly what LZ version you have what happens then.

Edit: I also tried running a full thinking time bot as LZ-1a198f-t2-nolim, but it caused my computer to lockup and lose on time. Like my computer was totally dead and I had to cold power it.

gcp commented 6 years ago

From my home machine playing on CGOS is essentially impossible because I time out all the time. From the LZ webserver it's rock solid (but obviously that can't run full strength bots). I never understood why the difference is so big, it's not like my connection to other sites drops.

tl;dr timeouts on CGOS are a thing and a reason I don't like it for measuring strength.

jjoshua2 commented 6 years ago

@killerducky it does have a win on time and no losses, and is doing well enough that it must be getting enough gpu time. see http://www.yss-aya.com/cgos/19x19/cross/LZ-fc2b-1080ti-t4.html

killerducky commented 6 years ago

@jjoshua2 looks like you have the strongest LZ bot on CGOS. I'm thinking about creating a way to share information like command line, hardware, etc for various bots. Would you be interested? Choices would be: shared Google doc, senseis' library page, or... other ideas?

jjoshua2 commented 6 years ago

I like Google docs. But I put all the information in the title. It's just standard lz12 with a 1080ti with 4 threads on that latest net.

jjoshua2 commented 6 years ago

just lost Zen-11.4n-1c due to a bad ladder after beating top Zen-15.7-2c1g

killerducky commented 6 years ago

You might consider compiling from the next branch for your next CGOS bot. There have been a few recent commits that add more speed/strength. I'd like to see a bot on CGOS with best hardware, best code, best network.

Also there was a UCTNode refactor

jjoshua2 commented 6 years ago

I'm on windows and I don't have a build setup. If an exe was posted I would update whenever I update nets.

jjoshua2 commented 6 years ago

Can I just grab the latest exe from https://ci.appveyor.com/project/gcp/leela-zero and that will be it? I see next on there with a recent exe... Edit, i don't actually see any exe in the artificats like there is for leela-chess

jjoshua2 commented 6 years ago

Looks like #938 already addresses this, but its not an easy fix so there's no exe now...

killerducky commented 6 years ago

@jjoshua2 looks it did time out in a later game after only a few moves.

http://www.yss-aya.com/cgos/19x19/SGF/2018/03/08/392929.sgf

jjoshua2 commented 6 years ago

@killerducky see #994. if this is fixed in next I can just switch to that because I installed visual studio, but I don't want to run buggy code.