Patchwork 3.16.0 indexes forever when you make a post

[cinnamon-bun]

Doing anything that writes to your feed (such as liking a post) puts Patchwork into indexing mode forever. I think the write succeeds but you can't do any more while indexing mode is happening. Restart Patchwork and it seems ok, the write is there, but the next action will put it into indexing mode again.

Rolled back to 3.15.0 and didn't have this problem.

MacOS 10.11.6

[cinnamon-bun]

No errors in the developer console.

Got this in the terminal:

Error replicating with @o3OcykTU7zdsm+WKoGl74XAoyDI/SWlVbIeyHHinQXQ=.ed25519:
   Error: stream ended with:3 but wanted:9
    at drain (/Users/cinnamon/projects/ssb/patchwork/node_modules/pull-reader/index.js:43:26)
    at /Users/cinnamon/projects/ssb/patchwork/node_modules/pull-reader/index.js:63:18
    at /Users/cinnamon/projects/ssb/patchwork/node_modules/pull-reader/index.js:114:13
    at drain (/Users/cinnamon/projects/ssb/patchwork/node_modules/stream-to-pull-stream/index.js:126:18)
    at Socket.<anonymous> (/Users/cinnamon/projects/ssb/patchwork/node_modules/stream-to-pull-stream/index.js:143:5)
    at Socket.emit (events.js:187:15)
    at endReadableNT (_stream_readable.js:1092:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)

[christianbundy]

I'll snag Kathleen's laptop and see if I can reproduce over there. Sorry this new update broke stuff for you, I'll try to get it fixed ASAP.

[cinnamon-bun]

Turns out @o3OcykTU... is the pub many.butt.nz

[christianbundy]

I'll keep investigating, but if you have time could you try running from master to see whether this is a problem with the DMG that electron-builder generated?

[cinnamon-bun]

Just ran from master, verified same problem.

[christianbundy]

Hmm. Assuming you've already tried all of the usual stuff (restarting your computer, etc), maybe you could do a git bisect? It'll search the commits between now and the last working version. For example, using fake good/bad info:

$ git bisect start
$ git bisect bad
$ git bisect good v3.15.0
Bisecting: 16 revisions left to test after this (roughly 4 steps)
[6387d0b2c0ead3907aaaaba566daad249800b52b] Merge branch 'no-bulk-require'
$ npm ci && npm start
$ git bisect good
Bisecting: 7 revisions left to test after this (roughly 3 steps)
[a3a44a7404b81a82e5c5da9ded92a895f3d1c0f2] Merge pull request #1167 from ssbc/fix-show-gathering-when-created
$ npm ci && npm start
$ git bisect good
Bisecting: 3 revisions left to test after this (roughly 2 steps)
[5494e1375bc2e850feb60bc808c95dd373547df1] Fix profile preview for encoded hashtags
$ npm ci && npm start
$ git bisect bad
Bisecting: 1 revision left to test after this (roughly 1 step)
[ce5f2faeff42ad8bc301aa8904338c7b53d8095f] Fix style problems from StandardJS linter
$ npm ci && npm start
$ git bisect bad
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[e35d9605889f777c47583df323710897d7f96822] Upgrade deps, mostly for ssb-markdown
$ npm ci && npm start
$ git bisect bad
e35d9605889f777c47583df323710897d7f96822 is the first bad commit
commit e35d9605889f777c47583df323710897d7f96822
Author: Christian Bundy <christianbundy@fraction.io>
Date:   Tue Aug 20 11:09:32 2019 -0700

    Upgrade deps, mostly for ssb-markdown

    This also adds some code to handle URI-encoded links, which are used by
    ssb-markdown for emoji hashtags. #🙃

 lib/catch-links.js |   8 +++-
 package-lock.json  | 106 ++++++++++++++++++++++++++++++++++++++++-------------
 package.json       |  12 +++---
 3 files changed, 94 insertions(+), 32 deletions(-)

[christianbundy]

Just tried a Macbook Air with macOS 10.11.5 and I can't find a way to reproduce this bug yet. Hmm.

[cinnamon-bun]

I have to remove the spellchecker to compile it, so I can't do git bisect. Doing it by hand...

newest commit first

634722a1 (3.16.0): hangs in indexing mode 739b761a (Add changelog entries for latest changes): not tested ⚠️ 388f2928 (Upgrade spacetime, ssb-legacy-conn, and standard): hangs in indexing mode d0cbb7bd (Merge pull request #1169 from ssbc/fix-hashtag-profile-preview): good 81718436 (Merge pull request #1168 from ssbc/upgrade-ssb-markdown): good 4fe3a334 (Merge remote-tracking branch 'origin/master' into ssb-legacy-conn): good 33f96c4d (3.15.0): good

Found it!

(node version v10.8.0)

[cinnamon-bun]

When running with 388f292 I see this error in the terminal. I don't know if this is a regular problem or not

Error: The given address is not a valid multiserver-address
    at ConnStaging._assertValidAddress (/Users/rabbit/projects/ssb/patchwork/node_modules/ssb-conn-staging/lib/index.js:32:19)
    at ConnStaging.unstage (/Users/rabbit/projects/ssb/patchwork/node_modules/ssb-conn-staging/lib/index.js:58:14)
    at Object.Gossip.remove (/Users/rabbit/projects/ssb/patchwork/node_modules/ssb-legacy-conn/lib/gossip.js:216:30)
    at Object.<anonymous> (/Users/rabbit/projects/ssb/patchwork/node_modules/secret-stack/util.js:80:17)
    at Object.<anonymous> (/Users/rabbit/projects/ssb/patchwork/node_modules/hoox/index.js:3:17)
    at Object.hooked [as remove] (/Users/rabbit/projects/ssb/patchwork/node_modules/hoox/index.js:10:15)
    at ssb.gossip.peers.slice.forEach (/Users/rabbit/projects/ssb/patchwork/lib/plugins/index.js:100:24)
    at Array.forEach (<anonymous>)
    at patchwork.contacts.raw.get (/Users/rabbit/projects/ssb/patchwork/lib/plugins/index.js:94:34)
    at get (/Users/rabbit/projects/ssb/patchwork/node_modules/flumeview-reduce/inject.js:118:11)

I'm guessing this is related to ssb-legacy-conn

[cinnamon-bun]

The address which is not a valid multiserver-address is @vYqLJ+S8RSwrgU6Nxja0kM3d19oWqjv9Og2JCbDd8+U=.ed25519 which is the infamous "@sbot" spambot. I have already blocked that feed.

[cinnamon-bun]

That address (of @sbot) is not in my gossip.json. I don't know where it's coming from.

Rolling back to the last version that works, d0cbb7bd, I see a slightly different error in the console, also from ssb-legacy-conn. This error doesn't seem to break anything. This address IS present in my gossip.json.

Error: not a valid address:{"port":8008,"key":"@J+0DGLgRn8H5tVLCcRUfN7NfUcTGEZKqML3krEOJjDY=.ed25519"}
    at Object.<anonymous> (/Users/rabbit/projects/ssb/patchwork/node_modules/ssb-legacy-conn/lib/index.js:287:31)
    at apply (/Users/rabbit/projects/ssb/patchwork/node_modules/muxrpc-validation/index.js:197:15)
    at Object.<anonymous> (/Users/rabbit/projects/ssb/patchwork/node_modules/muxrpc-validation/index.js:62:14)
    at Object.<anonymous> (/Users/rabbit/projects/ssb/patchwork/node_modules/secret-stack/util.js:80:17)
    at Object.<anonymous> (/Users/rabbit/projects/ssb/patchwork/node_modules/hoox/index.js:3:17)
    at Object.hooked [as add] (/Users/rabbit/projects/ssb/patchwork/node_modules/hoox/index.js:10:15)
    at /Users/rabbit/projects/ssb/patchwork/lib/plugins/index.js:145:26
    at /Users/rabbit/projects/ssb/patchwork/node_modules/pull-stream/sinks/drain.js:24:37
    at /Users/rabbit/projects/ssb/patchwork/node_modules/pull-stream/throughs/through.js:19:9
    at /Users/rabbit/projects/ssb/patchwork/node_modules/pull-stream/throughs/map.js:19:9

[cinnamon-bun]

My best guess is: this bug isn't about the OS version, it's triggered by something about your feed or local database.

Should I try deleting some indexes or something?

[christianbundy]

I'm also seeing the "Error: not a valid address", not sure what that's about. I'll plan to downgrade ssb-legacy-conn to 1.x.x, hopefully that will fix it? I'll make a PR now.

Maybe relevant: https://github.com/staltz/ssb-legacy-conn/issues/3

cc: @staltz

[cinnamon-bun]

I still have 50,000 messages from @sbot in my feed though I have blocked them. Is this related to deletion of blocked feeds?

[christianbundy]

Did you enable the setting that deletes messages from blocked feeds? IIRC after enabling you have to either block someone or restart Patchwork to trigger a delete.

[cinnamon-bun]

Nope, I haven't yet enabled that setting

[cinnamon-bun]

Tried starting from master and downgrading ssb-legacy-conn to ^1.0.0 which is currently 1.0.25. Didn't help. (Same thing as the PR you just made)

Then I downgraded specifically to 1.0.12, the version we were using before our bad commit 388f292. ✨ it works!

So something happened in ssb-legacy-conn between 1.0.12 and 1.0.25.

[christianbundy]

:mag: :mag: :mag:

Nice detective skills! Just updated my PR, that's awesome that you were able to track down the problem so quickly.

[cinnamon-bun]

Great teamwork! I'll let you merge the PR (I don't know how to do a release build either)