global pinning: phase 3

[mortelli]

User story: Global Pinning

Issue for implementing Phase 3 of global pinning.

The purpose of this phase is to manually test how global pinning works in a swarm network and fix whatever issues may arise.

Tasks

• [x] add metrics and/or logs to the global pinning code in order to keep track of what's going on.
• [x] write a script to update (and maybe create) recovery feeds.
• [x] set up a network with a number of nodes, with a subset of them configured as global pinners.
• [x] execute a round of the testing steps successfully

Testing steps

Once all other tasks are complete, the outline of the testing steps is as follows:

1. boot up all the nodes in the network with a small storage capacity.
2. upload a small file f to the network with a given node p marked as its publisher in its manifest. 2.1 verify locally that the publisher is correctly reflected in the manifest.
3. fetch and locally pin f in all of the global pinner nodes.
4. verify that the f does not have its chunks garbage collected in the pinner nodes after uploading more content to them.
5. make f have its chunks garbage collected from the network by re-uploading content to the appropriate nodes. this means that when a non-pinner asks for this resource, it will fail to retrieve its chunks and trigger recovery requests when attempting to fetch them. 5.1 verify that f cannot be fetched from a non-pinner node. the node should not have f already synced to it.
6. use the publisher node p to create a recovery feed for f.
7. update the recovery feed content, setting it to prefixes of the global pinners addresses, as a list of targets. the length of the prefixes should be 1 byte.
8. pick a non-pinner node and attempt to fetch f. 8.1 verify that there was a failure to obtain the related chunks, trojan messages were mined and sent to the global pinners.
9. verify that the global pinners received trojan messages. 9.1 check that f has its chunks re-uploaded to the network from the pinner nodes.
10. after some time. re-try fetching f from a non-pinner node which does not already store it locally.

If step 10 is successful, the process works as expected.

PRs

#	Title	Status
#2169	condition trojan chunk inspection	closed
#2183	add fallback content recovery publisher	closed
#2185	enable content recovery requests all nodes	merged
#2188	improve global pinning recovery hook tests	merged
#2190	fix for pinned chunks that are GC under certain conditions	closed
#2191	condition trojan chunk inspection	merged
#2194	repair globally pinned content through CLI publisher	closed
#2197	split sync flag into push and pull flags	merged
#2199	use bzzaccounts as global pinning publishers	merged
#2200	local pinning fix for pinned chunks	merged
#2201	further condition trojan chunk inspection	merged
#2202	add query param for publisher and solve flooding in recovery	merged
#2203	remove putToGCCheck calls in localstore	merged
#2208	add global pinning logs	merged

[zelig]

1. take a cluster of nodes. with tiny storage capacity
2. create file = chunk_0, chunk_1 , ... chunk_n aand file/i=chunk0, ....chunk{i-1}, chunk_{i+1},....chunk_n
3. upload file/i to the network, then pin file locally with nodes n_0, ..., n_k
4. test random node can retrieve file
5. keep uploading files and at the same time retrieve file/i
6. monitor debug to check if node closest to chunk_i garbage collects
7. random node m tries downloading and falis several times
8. publish feed update
9. m download successfully
10. other random nodes download successfully

[mortelli]

external feed update script here

[mortelli]

5. make f have its chunks garbage collected from the network (how do we do this?); this means that when a non-pinner asks for this resource, it will fail to retrieve its chunks and trigger recovery requests when attempting to fetch them. 5.1 verify that f cannot be fetched from a non-pinner node. the node should not have f already synced to it (how do we make sure of this?)

as @zelig said, @vojtechsimetka also suggested having nodes with very little storage so that the chunks for f are garbage collected when something else is uploaded.

we should test whether sync restores this content after having it replaced with something new is put into the nodes.

7. update the recovery feed content, setting it to prefixes of the global pinners addresses, as a list of targets (how do we determine the length of the prefixes?)

this one is still an open question

[mortelli]

this one is still an open question

1 byte appears to be enough for a small network.

[mortelli]

we're running tests on the global-pinning-test branch.

logs and testing funcs are being added in a grotesque manner to help us out during this manual testing phase.

after succeceding, we expect to:

• extract the useful logs into the global-pinning-logs branch and make a PR for it
• delete this testing branch
• look into automating these manual tests

[mortelli]

Latest status here

[mortelli]

GP POC has been ended.

Further issues can be tackled separately if needed.