application-research / autoretrieve

A server to make GraphSync data accessible on IPFS
22 stars 7 forks source link

feat: confirm retrievals resulted in a block in the blockstore #121

Closed rvagg closed 2 years ago

rvagg commented 2 years ago

Upon successful retrieval, provide a confirmation along with the reported event that the retrieval resulted in the block being stored in the blockstore. Using retrievedCids as a proxy isn't enough because it's possible (with some non-trivial probability) that a retrieval takes place for a block that's just been retrieved as part of a parallel retrieval, so graphsync doesn't need to transfer any blocks.

The backstory for this is that we're seeing evidence of cases where a retrieval might look successful but doesn't result in any blocks being transferred that can't be explained by the block having already been in the blockstore. So now we get a retrievedCids int and a confirmed bool on the events. When we see a confirmed of false for a "success" event then it's a bug. The intended behaviour of the stack is that "success" is always confirmed.

rvagg commented 2 years ago
Screenshot 2022-09-09 at 11 44 26 am

Already seeing some unconfirmed in my local instance on this branch.

codecov-commenter commented 2 years ago

Codecov Report

Base: 13.98% // Head: 13.92% // Decreases project coverage by -0.05% :warning:

Coverage data is based on head (da797f2) compared to base (fa63208). Patch coverage: 14.28% of modified lines in pull request are covered.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #121 +/- ## ========================================== - Coverage 13.98% 13.92% -0.06% ========================================== Files 16 16 Lines 2217 2226 +9 ========================================== Hits 310 310 - Misses 1892 1901 +9 Partials 15 15 ``` | [Impacted Files](https://codecov.io/gh/application-research/autoretrieve/pull/121?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=application-research) | Coverage Δ | | |---|---|---| | [autoretrieve.go](https://codecov.io/gh/application-research/autoretrieve/pull/121/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=application-research#diff-YXV0b3JldHJpZXZlLmdv) | `0.00% <0.00%> (ø)` | | | [filecoin/retriever.go](https://codecov.io/gh/application-research/autoretrieve/pull/121/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=application-research#diff-ZmlsZWNvaW4vcmV0cmlldmVyLmdv) | `0.00% <0.00%> (ø)` | | | [filecoin/retrieverevents.go](https://codecov.io/gh/application-research/autoretrieve/pull/121/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=application-research#diff-ZmlsZWNvaW4vcmV0cmlldmVyZXZlbnRzLmdv) | `0.00% <0.00%> (ø)` | | | [filecoin/eventrecorder/eventrecorder.go](https://codecov.io/gh/application-research/autoretrieve/pull/121/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=application-research#diff-ZmlsZWNvaW4vZXZlbnRyZWNvcmRlci9ldmVudHJlY29yZGVyLmdv) | `84.00% <100.00%> (ø)` | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=application-research). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=application-research)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.