No reconcile between peaks and SKS

[skasks]

Hi, just compiled peaks and let it run with an almost empty database - you need at least one key to get going.

It looks like that peaks closes the socket upon a EAGAIN.

the SKS server says: Aug 13 10:44:14 sv-2s127b sks[4210]: 2019-08-13 10:44:14 Beginning recon as server, client: <ADDR_INET [10.20.40.205]:37724> Aug 13 10:44:14 sv-2s127b sks[4210]: 2019-08-13 10:44:14 Joining reconciliation Aug 13 10:45:22 sv-2s127b sks[4210]: 2019-08-13 10:45:22 Reconciliation failed. Returning elements returned so far: Sys_error("Broken pipe") Aug 13 10:45:22 sv-2s127b sks[4210]: 2019-08-13 10:45:22 Reconciliation complete

peaks says:

5689 connect(5, {sa_family=AF_INET, sin_port=htons(11370), sin_addr=inet_addr("10.20.40.169")}, 16) = 0 5689 sendto(5, "\0\0\0~\n\0\0\0\5\0\0\0\7version\0\0\0\0051.1.6\0\0\0\thttp port\0\0\0\4\0\0,m\0\0\0\nbitquantum\0\0\0\4\0\0\0\2\0\0\0\4mbar\0\0\0\4\0\0\0\5\0\0\0\7filters\0\0\0\33yminsky.dedup,yminsky.merge", 130, MSG_NOSIGNAL, NULL, 0) = 130 5689 recvfrom(5, "\0\0\0~", 4, MSG_WAITALL, NULL, NULL) = 4 5689 recvfrom(5, "\n\0\0\0\5\0\0\0\nbitquantum\0\0\0\4\0\0\0\2\0\0\0\7filters\0\0\0\33yminsky.dedup,yminsky.merge\0\0\0\thttp port\0\0\0\4\0\0,k\0\0\0\4mbar\0\0\0\4\0\0\0\5\0\0\0\7version\0\0\0\0051.1.6", 126, MSG_WAITALL, NULL, NULL) = 126 5689 sendto(5, "\0\0\0\6passed", 10, MSG_NOSIGNAL, NULL, 0) = 10 5689 recvfrom(5, "\0\0\0\6", 4, MSG_WAITALL, NULL, NULL) = 4 5689 recvfrom(5, "passed", 6, MSG_WAITALL, NULL, NULL) = 6 5689 setsockopt(5, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0 5689 setsockopt(5, SOL_TCP, TCP_KEEPIDLE, [1], 4) = 0 5689 setsockopt(5, SOL_TCP, TCP_KEEPINTVL, [1], 4) = 0 5689 setsockopt(5, SOL_TCP, TCP_KEEPCNT, [60], 4) = 0 5689 setsockopt(5, SOL_SOCKET, SO_RCVTIMEO, "\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 16) = 0 5689 setsockopt(5, SOL_SOCKET, SO_SNDTIMEO, "\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 16) = 0 5689 recvfrom(5, "\0\0\0w", 4, MSG_WAITALL, NULL, NULL) = 4 5689 recvfrom(5, "\0\0\0\0\0\0\0\0\0\0Y\274h\0\0\0\6GSDG\302\205\4\313G\33T\3671\245\24I\0\235\367\221\363\260\241:\222Uw<m\22\351!\241\0\233\250@\351\v\24C\336\222\177t\341;\17\3423\0\0021\205\\353\300\344\375o\241\206\201\261V7C\1`|\242\217GF\10\311\325;\264\331\277\237\3279\1\346\323$\21\375\350\217\222x9\366\20\21s;\6\0", 119, MSG_WAITALL, NULL, NULL) = 119 5689 sendto(3, "\17\0\0\0\27\10\0\0\0\0\1\0\0\0\0\1\376\0\0", 19, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 19 5689 recvfrom(3, 0x55e73649a330, 16384, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable) 5689 poll([{fd=3, events=POLLIN}], 1, -1) = 1 ([{fd=3, revents=POLLIN}]) 5689 recvfrom(3, "\1\0\0\1\5=\0\0\2\3def\rgpg_keyserver\5ptree\5ptree\10node_key\10node_key\f\10\0\310\0\0\0\375\3P\0\0\0E\0\0\3\3def\rgpg_keyserver\5ptree\5ptree\fnode_svalues\fnode_svalues\f?\0\377\377\0\0\374\221\20\0\0\0E\0\0\4\3def\rgpg_keyserver\5ptree\5ptree\fnum_elements\fnum_elements\f?\0\v\0\0\0\3\1\20\0\0\0005\0\0\5\3def\rgpg_keyserver\5ptree\5ptree\4leaf\4leaf\f?\0\1\0\0\0\1\1\20\0\0\0G\0\0\6\3def\rgpg_keyserver\5ptree\5ptree\rnode_elements\rnode_elements\f?\0\377\377\0\0\374\221\20\0\0\0\5\0\0\7\376\0\0\2\0\25\0\0\10\0\0\0\v1 1 1 1 1 1\0\0\0\0\1\0\5\0\0\t\376\0\0\2\0", 16384, MSG_DONTWAIT, NULL, NULL) = 391 5689 recvfrom(5, "\0\0\0\1", 4, MSG_WAITALL, NULL, NULL) = 4 5689 recvfrom(5, "\6", 1, MSG_WAITALL, NULL, NULL) = 1 5689 sendto(5, "\0\0\0\5\3\0\0\0\0", 9, MSG_NOSIGNAL, NULL, 0) = 9 5689 recvfrom(5, 0x7f58acd2c4ec, 4, MSG_WAITALL, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable) 5689 shutdown(5, SHUT_RDWR) = 0 5689 shutdown(-1, SHUT_RDWR) = -1 EBADF (Bad file descriptor) 5689 close(5) = 0 5689 close(-1) = -1 EBADF (Bad file descriptor) 5689 nanosleep({tv_sec=60, tv_nsec=0}, 0x7f58acd2c6b0) = 0 5689 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 5 5689 stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=724, ...}) = 0 5689 openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 7 5689 lseek(7, 0, SEEK_CUR) = 0 5689 fstat(7, {st_mode=S_IFREG|0644, st_size=233, ...}) = 0 5689 read(7, "\n", 4096) = 233 5689 lseek(7, 0, SEEK_CUR) = 233 5689 read(7, "", 4096) = 0 5689 close(7) = 0 5689 shutdown(-1, SHUT_RDWR) = -1 EBADF (Bad file descriptor) 5689 shutdown(5, SHUT_RDWR) = -1 ENOTCONN (Transport endpoint is not connected) 5689 close(-1) = -1 EBADF (Bad file descriptor) 5689 close(5) = 0 5689 nanosleep({tv_sec=60, tv_nsec=0},

[r4yan2]

This is odd, on syslog is recorded something? If you have the possibility could you please make a test run with debug output? (--debug, and --stdout if you prefer to have the output on console instead of syslog)

[skasks]

peaks_recon_daemon[1758]: starting gossip client peaks_recon_daemon[1758]: starting gossip server peaks_recon_daemon[1758]: choose as partner: 10.20.40.169 peaks_recon_daemon[1758]: binding ok, proceed to listen for incoming connections peaks_recon_daemon[1758]: choosen partner 10.20.40.169 peaks_recon_daemon[1758]: Could not interpolate because size_diff (5880936) > size of values(6)! peaks_recon_daemon[1758]: current value of n: 0 peaks_recon_daemon[1758]: current value of n: 0 peaks_recon_daemon[1758]: IOException on recv, sorry but it is normal during recon as server peaks_recon_daemon[1758]: Resource temporarily unavailable peaks_recon_daemon[1758]: going to sleep... peaks_recon_daemon[1758]: ...resuming gossip peaks_recon_daemon[1758]: starting gossip client peaks_recon_daemon[1758]: choose as partner: 10.20.40.169 peaks_recon_daemon[1758]: choosen partner 10.20.40.169 peaks_recon_daemon[1758]: Could not interpolate because size_diff (5880936) > size of values(6)! peaks_recon_daemon[1758]: current value of n: 0 peaks_recon_daemon[1758]: current value of n: 0 peaks_recon_daemon[1758]: IOException on recv, sorry but it is normal during recon as server peaks_recon_daemon[1758]: Resource temporarily unavailable peaks_recon_daemon[1758]: going to sleep...

[skasks]

I'm adding some keys to the database of peaks now. But the output stays the same. The DB contains now:

| 3365 | selfSignaturesMetadata | | 84089 | gpg_keyserver | | 0 | KeyStatus | | 4870 | ptree | | 0 | removed_hash |

[r4yan2]

Well...it was working some commits ago, so maybe I've introduced some bug in a refactor, will check asap

[skasks]

I was importing all the keys, but got a OOM of mariadb and a corrupted DB. So I restarted with a fresh DB. Using import --fast, then build, then unpack & analyze

Tables are filling up: +------------+-------------------------+ | TABLE_ROWS | TABLE_NAME | +------------+-------------------------+ | 28878 | selfSignaturesMetadata | | 5854540 | gpg_keyserver | | 0 | KeyStatus | | 353154 | ptree | | 0 | removed_hash | | 0 | broken_keys | | 26447 | UserID | | 39020 | Pubkey | | 66294 | Signatures | | NULL | Signature_no_issuing_fp | | 661 | Unpacker_errors | | 1459 | revocationSignatures | | 0 | SignatureStatus | | 254 | UserAttribute | | NULL | key_primary_userID | +------------+-------------------------+

The error remains unchanged for recon: peaks_recon_daemon[23829]: ...resuming gossip peaks_recon_daemon[23829]: starting gossip client peaks_recon_daemon[23829]: choose as partner: 10.20.40.169 peaks_recon_daemon[23829]: choosen partner 10.20.40.169 peaks_recon_daemon[23829]: Could not interpolate because size_diff (5880936) > size of values(6)! peaks_recon_daemon[23829]: current value of n: 0 peaks_recon_daemon[23829]: current value of n: 0 peaks_recon_daemon[23829]: IOException on recv, sorry but it is normal during recon as server peaks_recon_daemon[23829]: Resource temporarily unavailable peaks_recon_daemon[23829]: going to sleep...

[r4yan2]

maybe related to the server and client daemon both running at the same time Try the following flow from a clean setup

peaks import --fastimport
peaks build
peaks recon --client-only

or just the third row, if you want to import all from sks, but you need to call the command twice, because peaks will stop with a warning the first time you attempt to start the recon with an empty tree

[skasks]

I pulled the repository to

dropped the database, imported one (single) key via fastimport, ran build, then:

./peaks recon --client-only --debug --stdout searching config file peaks_config config file found! Starting recon_daemon logging to stdout debug output starting only as client peaks_recon_daemon[2861]: starting gossip client peaks_recon_daemon[2861]: choose as partner: 10.20.40.169 peaks_recon_daemon[2861]: Sending Peer config peaks_recon_daemon[2861]: Receiving Peer config peaks_recon_daemon[2861]: Sending Config ok peaks_recon_daemon[2861]: choosen partner 10.20.40.169 peaks_recon_daemon[2861]: Receiving Recon Request Poly for peaks_recon_daemon[2861]: Could not interpolate because size_diff (5880935) > size of values(6)! peaks_recon_daemon[2861]: current value of n: 0 peaks_recon_daemon[2861]: current value of n: 0 peaks_recon_daemon[2861]: Sending Full Elements peaks_recon_daemon[2861]: IOException on recv, sorry but it is normal during recon as server peaks_recon_daemon[2861]: Resource temporarily unavailable peaks_recon_daemon[2861]: going to sleep... peaks_recon_daemon[2861]: ...resuming gossip

The sks server: Sep 3 09:34:00 sv-2s127b sks[461]: 2019-09-03 09:34:00 Beginning recon as server, client: <ADDR_INET [10.20.40.205]:48614> Sep 3 09:34:00 sv-2s127b sks[461]: 2019-09-03 09:34:00 Joining reconciliation Sep 3 09:35:09 sv-2s127b sks[461]: 2019-09-03 09:35:09 Reconciliation failed. Returning elements returned so far: Sys_error("Broken pipe") Sep 3 09:35:09 sv-2s127b sks[461]: 2019-09-03 09:35:09 Reconciliation complete