application-research / autoretrieve

A server to make GraphSync data accessible on IPFS
22 stars 7 forks source link

feat: add more metrics #125

Closed rvagg closed 1 year ago

rvagg commented 1 year ago

Here's what I'm imagining for the dashboard with the addition of these metrics:

  1. Measurement of the full pipeline from bitswap to retrieval, concentric circles getting smaller as we progress. a. BitswapRetrieverRequestCount (existing) is the total CIDs that we're attempting b. BitswapRequestWithIndexerCandidatesCount is the number that we have candidates for c. BitswapRequestWithSuccessfulQueryCount is the number that we have at least one successful query response for d. RetrievalDealSuccessCount is the number that we manage to retrieve
  2. Additional data from the indexer and query phases: a. IndexerCandidatesCount tells us how many candidates we're getting from the indexer, we should be able to do things like divide it by BitswapRequestWithIndexerCandidatesCount to find the average number of candidates per found CID. b. RetrievalQueryCount is similar but for queries, we could divide by BitswapRequestWithSuccessfulQueryCount to get the average number of successful queries.
  3. Accounting for specific errors: there's an error for each of the current top error types we're seeing that we'll do a strings.Contains() match on the error message for to match & report. Plus there's a RetrievalErrorOtherCount for the remainder which we can monitor and if it gets high enough we can add more. I'm imagining a table of these with %'s of the total failure count for each. Not sure if Grafana can do that yet but I'm sure it can do something similar.
codecov-commenter commented 1 year ago

Codecov Report

Base: 13.70% // Head: 13.51% // Decreases project coverage by -0.18% :warning:

Coverage data is based on head (5758576) compared to base (1d2d50a). Patch coverage: 0.00% of modified lines in pull request are covered.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #125 +/- ## ========================================== - Coverage 13.70% 13.51% -0.19% ========================================== Files 16 16 Lines 2226 2257 +31 ========================================== Hits 305 305 - Misses 1905 1936 +31 Partials 16 16 ``` | [Impacted Files](https://codecov.io/gh/application-research/autoretrieve/pull/125?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=application-research) | Coverage Δ | | |---|---|---| | [filecoin/retriever.go](https://codecov.io/gh/application-research/autoretrieve/pull/125/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=application-research#diff-ZmlsZWNvaW4vcmV0cmlldmVyLmdv) | `0.00% <0.00%> (ø)` | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=application-research). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=application-research)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.