feat: generates parts of the dashboard async, using ipfs/gateway-conformance

laurentsenta commented 1 year ago

The ipfs/gateway-conformance has reached a point where we should be able to generate most of the info needed by our public gateway checker. Reusing the project should reduce maintenance and avoid the "Red Message Of Death" caused by the user's browser connecting with many gateways.

Interop

[x] Agree on the output format we want to share between conformance and the frontend

conformance.json:

{
  "gateway.ipfs.io": {
    "metadata": { "timestamp": ...., "country": .... },
    "results": {
          "CORS": { "success": 12, "fail": 0, "skipped": 0 },
          "IPNS": ...,
    }
   },
  "another-gateway.link": ...
}

CI: Call the gateway conformance workflow - https://github.com/ipfs/public-gateway-checker/pull/450

@laurentsenta

[x] Reuse the workflow from e2e production workflow
[x] Make the list of gateways that live in gateways.ts available to the workflow
[x] create a minimal output we can render in the CI
[x] Extract the results in a format that can be reused here
- [x] Export TestGroups metadata
- [x] Aggregates results with a piece of js

CI: Generate metadata (country) on the server side - https://github.com/ipfs/public-gateway-checker/issues/451

Frontend: Render the results

[ ] import jsons to render static results
- [ ] conformance.json
[ ] "Online" column - https://github.com/ipfs/public-gateway-checker/issues/452

Make the tests pass

[ ] likely we'll have issues with missing fixtures

Follow-ups

[ ] Create issues from notes in https://github.com/ipfs/public-gateway-checker/issues/442#issuecomment-1688390763
[ ] Close duplicate tasks from https://github.com/ipfs/public-gateway-checker/issues/442#issuecomment-1663344754

welcome[bot] commented 1 year ago

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review. In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment. Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

"Priority" labels will show how urgent this is for the team.
"Status" labels will show if this is ready to be worked on, blocked, or in progress.
"Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

laurentsenta commented 1 year ago

(self-assigning, I'll take the responsibility of moving this work forward, but happy to collaborate)

whizzzkid commented 1 year ago

Thanks for submitting this story @laurentsenta, excellent effort, we can pair on this if you'd like!

Can we mark these issues as duplicate/subtask of this issue (as I feel these can have a potential overlap):

CC: @SgtPooki

laurentsenta commented 1 year ago

@SgtPooki @whizzzkid I updated the plan and prepared the PR for the CI side.

@lidel currently the gateway check looks like this:

the conformance test suite granularity is much finer:

(from CI)

Proposal: we define TestGroups metadata in conformance testing that will be used in project like this one.

laurentsenta commented 1 year ago

note from call with @whizzzkid & @galargh

[ ] show a timestamp of the latest run time
[ ] take into account region-locked gateway, if any,
[ ] do we test badbits? could we test badbits in conformance?
- important: identifying gateways that don't host malicious content
[ ] the grand vision: might be used as bootstrap nodes.
[ ] We'll need to retry gateway checks
[ ] metadata: we ping the gateway, then load the geolocation
- output the IP out of the gway conformance or generate during ci
[ ] historical data: create automated issue when the gateway is failing to ping the author of the PR
- from experience: https://github.com/ipfs/public-gateway-checker/pull/443 lead to https://github.com/ipfs/public-gateway-checker/pull/444

BigLep commented 1 year ago

Thanks for updating @laurentsenta . A couplde of drive-by comments (don't block on me)...

In https://github.com/ipfs/public-gateway-checker/issues/442#issuecomment-1687950302 you took a screenshot including Gateway implementations rather than gateway instances. I assume that was just for convenience right? This issue concerns conformance metadata for gateway instances right?

Lots of interesting ideas in https://github.com/ipfs/public-gateway-checker/issues/442#issuecomment-1688390763. Which of these are we actually committing to? I want to make sure we don't move the goal post. Most (all?) those things seem like potential followups. I want us to get to a place where gateway checker red screen of death is gone and then we can evaluate what to do next. (I think at that point it will make sense to shift to higher impact areas.)

laurentsenta commented 1 year ago

@BigLep apologies, I forgot to press "send" on my answer,

In https://github.com/ipfs/public-gateway-checker/issues/442#issuecomment-1687950302 you took a screenshot including Gateway implementations rather than gateway instances. I assume that was just for convenience right? This issue concerns conformance metadata for gateway instances right?

Yes, this is in relation to a discussion regarding metadata in gateway conformance (comment). The screenshot points out the difference between gateway conformance details (10+ groups) vs gateway checker level of details (4 test groups). Yesterday we ack'd with @lidel the need for a "Taxonomie for End Users" that is different from the "Taxonomie for Specs Maintainers".

Lots of interesting ideas in https://github.com/ipfs/public-gateway-checker/issues/442#issuecomment-1688390763. Which of these are we actually committing to? I want to make sure we don't move the goal post. Most (all?) those things seem like potential followups.

Definitely. I'm working on matching features with the current workflow, the rest is nice-to-have or follow-up

ipfs / public-gateway-checker