search

ssbc / ssb-ebt

secure scuttlebutt replication with epidemic-broadcast-trees
MIT License
18 stars 10 forks source link

No replication despite connections #77

Closed staltz closed 1 year ago

staltz commented 1 year ago

I had a scary moment when my Manyverse was open for 5 min and no new data came in, despite being connected to 5 room peers. Then I restarted the app, it reconnected to peers, and still no replication. I changed from 4G to WiFi, and no effect. I just left the app open for 5 more minutes, and then finally it replicated. Super weird.

I'm sure it was a replication problem (and not a "people aren't talking" problem) because I saw from the raw db list that the last synced msg was 7 hours ago, and in SSB main net, there's new data with a higher frequency.

I have to try to reproduce it again.

Manyverse master branch on October 18 on Android

staltz commented 1 year ago

I managed to reproduce it again.

Then I installed an older version of Manyverse (0.2210.3) and the bug disappeared (i.e. replication happened immediately). This means that the bug was inserted somewhere between these commits

Screenshot from 2022-10-19 12-19-27@2x

I'll make an easy assumption that the culprit is https://github.com/staltz/manyverse/commit/22ca11a5b6ba679f8235725cf1e9af7bca87976f which updated ssb-db2 to 6.x (which uses feedFormats) and also a couple other package updates, like ssb-ebt 8.2.0 => 9.1.2.

I'll do a bit more fiddling with this, and see if I can use different versions of ssb-ebt until I find the commit in ssb-ebt where these things stopped working.

staltz commented 1 year ago

I reproduced the bug with ssb-ebt 9.0.0, so the bug must have been introduced between 8.2.1 and 9.0.0, which can only mean this commit: https://github.com/ssbc/ssb-ebt/commit/5a4dd39642c2969dde7f57d42e534d275f3d1cb2

staltz commented 1 year ago

Manyverse 0.2210.3 (the build that does NOT have this bug) uses epidemic-broadcast-trees 9.0.3, so we can rule out epidemic-broadcast-trees as the culprit.

staltz commented 1 year ago

So far it seems that privateIndex offset is at -1 when starting up the app, so it initiates indexing ssb-db2 from the first record, and ssb-replication-scheduler waits for indexing to complete before requesting anything in ebt. I wonder if there's a problem with saving private index results to disk.

arj03 commented 1 year ago

Strange

staltz commented 1 year ago

I'm pretty sure the problem is with the private index, I just ran out of time to work on this today so I'll continue tomorrow.

  1. Private index doesn't update the status, and it should (this would mean a progress bar would be shown in manyverse, currently no progress bar is shown so it seems like the app is idle)
  2. Private index is being rebuilt from scratch every single time I start up the app
staltz commented 1 year ago

Closing because... not an ssb-ebt issue and it was solved already in https://github.com/ssbc/ssb-db2/pull/399