search

kappa-db / multifeed

Multi-writer hypercore.
135 stars 27 forks source link

Truncated or corrupted headers #6

Closed cinnamon-bun closed 5 years ago

cinnamon-bun commented 5 years ago

So headers are now JSON length as a raw 32 bit integer + JSON string

It sometimes seems to start deserializing a header in the middle of the JSON string instead of at the beginning. Thus it gets a bogus very large length, and reads past the end of the JSON string:

2018-10-20T22:29:57.214Z multifeed [SERIALIZE] header len to read: 1664508262
2018-10-20T22:29:57.214Z multifeed [SERIALIZE] json buf to read: e4f620ff1849b360332a072397d5a4c16","ddd6e9f5dc08650a8a5e5a1ce3990629d29f94352ea1e84eacb90076439de19f","28ad933b863a0a46598f1760544310d59737d0b48ee22fa04194d8cea878400d","9595c19eebd2561b8ddc11693fcc665c4c692292236e312c840e08e917a33714","d75c00de316fe4e200f929b7ac0a281fe01a0f08d8790fb9bee3f3fbc9d7ed19","e14d822c91fd09a66c73bf414ab4e4e76c320eabd4d7f14fe2a33e38721a06df","130c291f49bdfa756bdd105c313c949a5e2c22774cfb3efa145208a163958078","1b60120774ca45c82c9bc3a4ec4be0fac4914a7e175f13425f7eac8e14991984"]}#�
 �t�1�!V��E��q�ֱ+B�z�s�T�S�r'
 ���������A���a��O�q}�@G��1C(��
2018-10-20T22:29:57.214Z multifeed [REPLICATION] recv'd header: {}
2018-10-20T22:29:57.214Z multifeed [REPLICATION] aborting (bad header)

Note the JSON that's being printed starts in the middle of a JSON object.

This happens reliably when joining the public cabal, starting with an empty local database: DEBUG=multifeed node_modules/.bin/cabal --key cabal://bd45fde0ad866d4069af490f0ca9b07110808307872d4b659a4ff7a4ef85315a --seed > foo 2>&1

After giving up on this peer it continues to sync with other peers, but there are several peers in this cabal which seem to have this problem every time. I thought they might be running old software from before the header was JSON, but that doesn't explain the JSON fragment we get in the header content.