Bug: Loading/Syncing stores on first load, when pinning server first opens store

zachferland commented 5 years ago

Mostly placeholder, until details are added. I have seen in the example. Looks like @oed has seen it and @michaelsena.

zachferland commented 5 years ago

Going to reword this issue. Not related to clear cache, just more obvious when clear. Happens when the pinning server first opens a db. All request once the db is already open don't have any problem.

The orbit db stores are not sharing all the heads or something, on first load/open only the first three logs entries are shared and then no more come through. Once you open again all the remaining are shared.

oed commented 5 years ago

Oh, It's that thing again. I thought I had solved that once 🙃

zachferland commented 5 years ago

Have been able to reliably reproduce this. Only happens on old accounts. Maybe from when we had issues with the iframe, it seems that we somehow 'corrupted' the the lower level data struct or something similar. Basically got orbitdb in some state that it cant handle it. So this is likely lower level, probably something we do in orbitdb to prevent this state or handle it. Since this is the case, its likely not worth prioritizing a fix at the moment. New accounts works, and old accounts work still if you open twice, no data loss.

Other notes

has entries does return the correct value on first pinning server open
on first pinning server open, we do get some log entries, and replicated is called. In my example i would always get 3 entries before it stopped syncing any more. Since this val has been consistent this does make it seem it could be some data struct thing, rather than some other async thing.
second pinning server open always works and fetch the remaining log entries. Why does it work all fine the second time? does it skip some log entries and then continue fine?

Would have to dig deeper into orbitdb to find out exactly what is happening.

zachferland commented 5 years ago

bumping, chatted about similar issue today, have to see if this looks like the same. If only older account or a handful, will have weigh tradeoffs of fixing, especially if same as described above.

michaelsena commented 5 years ago

It's not just older accounts, as new accounts i recently created with Fortmatic and Portis with the newest release also have the same issue.

zachferland commented 5 years ago

ok, the release on master? I haven't been able to recreate with any new account yet

michaelsena commented 5 years ago

Yeah the web3connect release that we recently pushed to master. But Kenzo has pushed a small fix for one of the syncing issues so let me try with cleared cache again to confirm.

michaelsena commented 5 years ago

@zachferland just confirmed this happened to me on first load of a new portis account (created 2 weeks ago). I cleared cache, and hadn't used it in a few days, so when I tried to sync data on first load it never connected to the pinning node. On second load, it synced the data fine.

zachferland commented 5 years ago

did it throw head hash doesn't match errors in the console?

michaelsena commented 5 years ago

No it didn't @zachferland

zachferland commented 5 years ago

yeah think it is different than before, error before was only cache clear, this seems to only be on first db open on pinning node, not on each cache clear

zachferland commented 5 years ago

can you see it on any new accounts used with metamask? instead of others

michaelsena commented 5 years ago

Can confirm it's on first DB open, and not because of cleared cache. Just tested that with portis.

Will try a new MM account, but need like an hour to create an account, then let the pinning node close and then try again.

oed commented 5 years ago

Ok, if that's the case we could test it faster with the services-box by setting the timeout to be shorter.

zachferland commented 5 years ago

yeah recreated locally now, this bug shouldn't be too bad, just have to find it now

zachferland commented 5 years ago

@oed not much to update so far, once revisiting today, could no longer recreate as mentioned above, have to really look to make sure i have the exact same env/builds, not sure what happened there

Did see 3boxjs not reconnecting to pinning node once the db has closed on the pinning node. Didn't get to look to deep yet, have further to go, but once db closed, peer stayed in this._ipfs.pubsub.peers, so pin db message was never sent again.

didn't look at anything related to head hashes yet

3box / 3box-js

Bug: Loading/Syncing stores on first load, when pinning server first opens store #324