status-im / nimbus-eth2

Nim implementation of the Ethereum Beacon Chain
https://nimbus.guide
Other
543 stars 233 forks source link

SIGSEGV on Pyrmont #2289

Closed tersec closed 3 years ago

tersec commented 3 years ago

https://github.com/status-im/nimbus-eth2/commit/614128654739102e6f4be6544864c57994358292

 peers: 160 ❯ finalized: 69e4b1ba:17155 ❯ head: 0a252491:17158:25 ❯ time: 17158:25 (549081) ❯ sync: synced
Thread 1 "nimbus_beacon_n" received signal SIGSEGV, Segmentation fault.
br_aes_x86ni_ctr_run (ctx=<optimized out>, iv=<optimized out>, cc=<optimized out>, data=0x7fffed9600b4, len=18446744073541955745)
    at nimbus-eth2/vendor/nim-bearssl/bearssl/csources/src/symcipher/aes_x86ni_ctr.c:158
--Type <RET> for more, q to quit, c to continue without paging--
158                             x1 = _mm_xor_si128(x1,
(gdb) bt
#0  br_aes_x86ni_ctr_run (ctx=<optimized out>, iv=<optimized out>, cc=<optimized out>, data=0x7fffed9600b4, len=18446744073541955745)
    at nimbus-eth2/vendor/nim-bearssl/bearssl/csources/src/symcipher/aes_x86ni_ctr.c:158
#1  0x0000555555ba38ff in do_ctr (xortag=0x7fffffffa920, len=18446744073709551585, data=0x7fffed9600b4, nonce=0x7fffed9600ac, 
    cc=0x7fffeef19300) at nimbus-eth2/vendor/nim-bearssl/bearssl/csources/src/ssl/ssl_rec_gcm.c:118
#2  gcm_encrypt (cc=0x7fffeef19300, record_type=21, version=771, data=0x7fffed9600b4, data_len=0x7fffffffa990)
    at nimbus-eth2/vendor/nim-bearssl/bearssl/csources/src/ssl/ssl_rec_gcm.c:206
#3  0x0000555555bb7786 in sendpld_flush (force=0, rc=0x7fffeef19048)
    at nimbus-eth2/vendor/nim-bearssl/bearssl/csources/src/ssl/ssl_engine.c:876
#4  sendpld_ack (rc=0x7fffeef19048, len=<optimized out>)
    at nimbus-eth2/vendor/nim-bearssl/bearssl/csources/src/ssl/ssl_engine.c:876
--Type <RET> for more, q to quit, c to continue without paging--
#5  0x0000555555a9d82e in br_ssl_engine_sendapp_ack (len=<optimized out>, cc=<optimized out>)
    at nimbus-eth2/vendor/nim-bearssl/bearssl/csources/src/ssl/ssl_engine.c:1133
#6  tlsWriteLoop__TxLhpDEp9aspRi1C4YYFQyQ (ClE_0=<optimized out>)
    at nimbus-eth2/vendor/nim-chronos/chronos/streams/tlsstream.nim:186
#7  0x0000555555a97364 in tlsWriteLoop_continue__Od6MLfJYG9bRfXGa6F1d0tA (udataX60gensym47491198_=<optimized out>, 
    ClE_0=<optimized out>) at nimbus-eth2/vendor/nim-chronos/chronos/asyncmacro2.nim:36
#8  0x0000555555b484e5 in poll__YNjd8fE6xG8CRNwfLnrx0g_2 () at nimbus-eth2/vendor/nim-chronos/chronos/asyncloop.nim:279
#9  0x000055555564deab in run__O3rBQRv7r7V2gs9cvIXh7BA_3 (node=node@entry=0x7ffff3a2f048)
    at nimbus-eth2/beacon_chain/nimbus_beacon_node.nim:1088
#10 0x00005555556501d2 in start__O3rBQRv7r7V2gs9cvIXh7BA_2 (node=<optimized out>)
--Type <RET> for more, q to quit, c to continue without paging--
    at nimbus-eth2/beacon_chain/nimbus_beacon_node.nim:1142
#11 0x000055555564ab87 in main__O177s9aQHqKldc12efy8Pbg () at nimbus-eth2/beacon_chain/nimbus_beacon_node.nim:1623
#12 0x00005555555b3264 in NimMain () at nimbus-eth2/vendor/nim-libp2p/libp2p/stream/bufferstream.nim:350
#13 main (argc=<optimized out>, args=<optimized out>, env=<optimized out>)
    at nimbus-eth2/vendor/nim-libp2p/libp2p/stream/bufferstream.nim:357
(gdb) 
tersec commented 3 years ago
(gdb) down
#0  br_aes_x86ni_ctr_run (ctx=<optimized out>, iv=<optimized out>, cc=<optimized out>, data=0x7fffed9600b4, len=18446744073541955745) at nimbus-eth2/vendor/nim-bearssl/bearssl/csources/src/symcipher/aes_x86ni_ctr.c:158
158                             x1 = _mm_xor_si128(x1,
(gdb) p buf
$1 = (unsigned char *) 0x7ffff7934ff4 ""
(gdb) l 159
154                     }
155                     if (len >= 64) {
156                             x0 = _mm_xor_si128(x0,
157                                     _mm_loadu_si128((void *)(buf +  0)));
158                             x1 = _mm_xor_si128(x1,
159                                     _mm_loadu_si128((void *)(buf + 16)));
160                             x2 = _mm_xor_si128(x2,
161                                     _mm_loadu_si128((void *)(buf + 32)));
162                             x3 = _mm_xor_si128(x3,
163                                     _mm_loadu_si128((void *)(buf + 48)));

One possible issue is that len is incorrect, leading it to believe it can access buf + 16:

(gdb) dow
#0  br_aes_x86ni_ctr_run (ctx=<optimized out>, iv=<optimized out>, cc=<optimized out>, data=0x7fffed9600b4, len=18446744073541955745)
    at nimbus-eth2/vendor/nim-bearssl/bearssl/csources/src/symcipher/aes_x86ni_ctr.c:158
158                             x1 = _mm_xor_si128(x1,
(gdb) p len
$12 = 18446744073541955745
(gdb) up
#1  0x0000555555ba38ff in do_ctr (xortag=0x7fffffffa920, len=18446744073709551585, data=0x7fffed9600b4, nonce=0x7fffed9600ac, 
    cc=0x7fffeef19300) at nimbus-eth2/vendor/nim-bearssl/bearssl/csources/src/ssl/ssl_rec_gcm.c:118
118             cc->bc.vtable->run(&cc->bc.vtable, iv, 2, data, len);
(gdb) p len
$13 = 18446744073709551585
(gdb) up
#2  gcm_encrypt (cc=0x7fffeef19300, record_type=21, version=771, data=0x7fffed9600b4, data_len=0x7fffffffa990)
    at nimbus-eth2/vendor/nim-bearssl/bearssl/csources/src/ssl/ssl_rec_gcm.c:206
206             do_ctr(cc, buf - 8, buf, len, tmp);
(gdb) p len
$14 = 18446744073709551585
(gdb) p (int64_t)len
$15 = -31

It seems to come from: https://github.com/status-im/nim-chronos/blob/dcf197a8c7891252fc611508a7d364d4953001d9/chronos/streams/tlsstream.nim#L169-L191

cheatfate commented 3 years ago

Supposed to be fixed in https://github.com/status-im/nim-chronos/pull/153 and bump https://github.com/status-im/nimbus-eth2/commit/5f924c0038899a58d91411366cd8446b360cb6c3