Open John-44 opened 1 year ago
John-44
commented
1 year ago Update: it has finally worked, history has now been successfully downloaded and unpacked! :-) So the issue to solve here is fixing whatever is preventing it from being started in a more timely manner
jrainville
commented
1 year ago Ah thanks for the update.
I am currently investigating, but I don't have full knowledge. At least now I know it can still work.
I'll try to figure out why it takes so long. I saw in my prod logs that I do receive the magnet links. I didn't have debug enabled on my dev one sadly, so I don't know yet if it received them.
I'll try to figure it out today, but if I can't, I'll just wait for Pascal to be back. Since it's not completely broken, it's not as big a priority as I figured.
jrainville
commented
1 year ago It triggered for me too. I created the account last Friday, didn't use it during the weekend, so it took ~10 hours.
I'll leave it to Pascal to try to shorten it if he can when he's back.
caybro
commented
1 year ago Just restored an account recently and for me it started almost immediately (in fact I could see the new messages signals in console even before the UI showed up)
John-44
commented
1 year ago Just restored an account recently and for me it started almost immediately (in fact I could see the new messages signals in console even before the UI showed up)
@caybro were you using 0.11 or Master?
Asking because I wonder if the recently fixed bug with fetching messages from waku store nodes may have fixed this? I think this bug has recently been fixed in master (and is definitely present in 0.11)
caybro
commented
1 year ago Just restored an account recently and for me it started almost immediately (in fact I could see the new messages signals in console even before the UI showed up)
@caybro were you using 0.11 or Master?
Asking because I wonder if the recently fixed bug with fetching messages from waku store nodes may have fixed this? I think this bug has recently been fixed in master (and is definitely present in 0.11)
I've been on master, so it looks like it indeed got fixed there
John-44
commented
1 year ago Just restored an account recently and for me it started almost immediately (in fact I could see the new messages signals in console even before the UI showed up)
@caybro were you using 0.11 or Master? Asking because I wonder if the recently fixed bug with fetching messages from waku store nodes may have fixed this? I think this bug has recently been fixed in master (and is definitely present in 0.11)
I've been on master, so it looks like it indeed got fixed there
Nice! But we should leave this issue open until QA have confirmed that this issue is fixed, because there is also a chance that fortuitous timing of when you chose to restore being the cause of the download starting immediately (need to test this multiple times, at different times, to be sure it is fixed).
I wonder if the "missing requests to join community" issue that @iurimatias has been facing is also connected to this (hopefully now fixed in master) waku message retrieval from store node bug? We shall see!
0x-r4bbit
commented
1 year ago I performed a clean fresh install of Status Desktop, and restored via seed phrase around 60 hours ago, Status Desktop has been running continually since then, and the community history archive download and unpack hasn't happened yet.
Either it is: A) delayed (in which case this needs to be fixed) or B) it is broken (in which case it also needs to be fixed)
There are various reasons why it might take some time for the archive protocol to kick in:
A look at your logs around that time should give us enough insights (I've put plenty of them in there).
I remember that previously we had an issue with the history download sometimes not starting for up to 7 days after a user joins a community. This issue is to track fixing this.
^ This is the case where a node receives a magnet link, tries to fetch the torrent info from the torrent network and fails to do so (it times out). When that happens, we retry again. If it still times out we just let it be and wait for the next magnet link, which is publish by the owner node as soon as it comes online again (or when it creates new archives).
If the owner node is (when online) is posting the torrent link to waku every say 8 hours (or if offline for more than 8 hours, it posts the torrent link to waku as soon as it comes online), then as long as the owner node has been online once in the last 30 days, a community member's install of Status Desktop should always be able to fetch the latest torrent link from waku store nodes at the same time as all the other initial fetching of messages from waku store nodes is performed. Is this correct? If so there must be a bug somewhere that is preventing this from working.
This is correct, except that right now the owner node does not republish a magnet link every 8 hours. It publishes it whenever it starts seeding a torrent for a community. This happens when:
a) The owner node comes online, and it already has a torrent file to be seeded b) The owner node creates new archives, updates the torrent file, and then seeds it
Due to a) this basically means, whenever the node comes online it'll publish a magnet link if it has a torrent to seed. Member nodes keep track of the last magnet link they've seen, so they ignore it if they've seen it before.
The download of the latest Community History Archive should start within 1 hour of a user joining a new community (or within 1 hour of a community member performing a fresh install of Status Desktop and restoring their communities via either syncing with another device or from the backup to waku)
Download of community archives will start as soon as a member node has received the torrent info from the magnet link. When joining a community, the community member will receive a magnet link along with its invite. If the torrent is available at that time, it will start the download (assuming it doesn't time out).
It's gonna be hard to put the 1 hour time constraint on that, because we never have the guarantee that someone is actually seeding the torrent.
I believe all in all, it'd be better to have some UI feedback for the user when loading of archives is happening. We can even show some progress with #n out of #m data pieces/archives so users get a feel for the progress...
jrainville
commented
1 year ago Moving to 0.13 as there is no easy fix doable in time for 0.12
iurimatias
commented
8 months ago moved to 2.29 due to lack of space in this milestone
Version 0.11, Windows 10
I performed a clean fresh install of Status Desktop, and restored via seed phrase around 60 hours ago, Status Desktop has been running continually since then, and the community history archive download and unpack hasn't happened yet.
Either it is: A) delayed (in which case this needs to be fixed) or B) it is broken (in which case it also needs to be fixed)
I remember that previously we had an issue with the history download sometimes not starting for up to 7 days after a user joins a community. This issue is to track fixing this.
If the owner node is (when online) is posting the torrent link to waku every say 8 hours (or if offline for more than 8 hours, it posts the torrent link to waku as soon as it comes online), then as long as the owner node has been online once in the last 30 days, a community member's install of Status Desktop should always be able to fetch the latest torrent link from waku store nodes at the same time as all the other initial fetching of messages from waku store nodes is performed. Is this correct? If so there must be a bug somewhere that is preventing this from working.
There was a bug with fetching historical messages older than 24h from waku store nodes that I think @richard-ramos has now fixed, I wonder if this bug was impacting the Community History Archive service?
Desired resolution: