search

ipfs / ipfs-companion

Browser extension that simplifies access to IPFS resources on the web
https://docs.ipfs.tech/install/ipfs-companion/
Creative Commons Zero v1.0 Universal
2.07k stars 324 forks source link

Use original gateway URL for content routing (async fetch of blocks and cars) #1042

Open askender opened 2 years ago

askender commented 2 years ago

tl;dr: https://github.com/ipfs/ipfs-companion/issues/1042#issuecomment-1081826521

Describe the bug and To Reproduce If we are using ipfs-companion, some gateway url cannot be openned. It only try to find the content at local ipfs node, never retry the gateway url itself.

To Reproduce Steps to reproduce the behavior:

  1. set ipfs-companion well for Chrome, and running a local ipfs daemon.
  2. open https://bafybeiae5c2ip2xxd5vvbnteu2sxntnmvgqqyt3we2s3hzbbcfifdsz3ya.ipfs.infura-ipfs.io/ using Chrome, the url cannot be opened. (It can be openned because someone pinned it now.)
  3. Close the ipfs-companion, then retry to open https://bafybeiae5c2ip2xxd5vvbnteu2sxntnmvgqqyt3we2s3hzbbcfifdsz3ya.ipfs.infura-ipfs.io/ again, it can be opened now.
  4. It seems the CID is not added to DHT, a ipfs gateway url with cid cannot opened. But the url is ok without ipfs-companion
  5. any options of ipfs-companion cannot fix this problem.

Expected behavior A url like https://bafybeiae5c2ip2xxd5vvbnteu2sxntnmvgqqyt3we2s3hzbbcfifdsz3ya.ipfs.infura-ipfs.io/ can be opened even if it is not in local ipfs node.

Screenshots No.

Desktop (please complete the following information):

askender commented 2 years ago

Detail: https://community.infura.io/t/do-you-need-an-infura-account-to-use-infura-ipfs/3422/2 I didn't use the account yet. As doc said https://infura.io/docs/ipfs#section/Authentication/Using-Javascript with a new Infura Authorization header. Everything else stays the same, as the Infura API is fully compatible with IPFS. So, Option1: without auth, Infura don't pin data in DHT, and even we use their http-gateway, if people use ipfs companion, that gateway url never opened, if people don't use ipfs companion, that gateway-url can be opened, everything is fine. So embarrassing. (But it seems it can be solved in ipfs-companion side. A timeout and retry the gateway url itself. ) Option2, we pay for Infura ipfs service, but because we running in browser and not web2 backend, it seems other people can use our account. Option3, we should find another free gateway which will auto-pin, or we host a ipfs gateway service ourself. As we are in China, host ipfs gateway maybe dangerous for ourselves.

So we still think ipfs-companion can solve the problem by retry the gateway url itself after fail to found in local ipfs node. (use gateway whitelist set by user, not only one default gateway)

Thanks!

SgtPooki commented 2 years ago

@lidel to respond on here with further information

lidel commented 2 years ago

@askender I think you should contact Infura and explain your needs to them:

As for what we can do in IPFS Companion – we won't add any special handling per gateway. Gateways are generic interface that aims to remove vendor lock-in. We also can't silently decrease integrity guarantees: users choose to use local IPFS gateway to ensure hash of every block and CID is verified.

Potential feature request: async fetch of block and CAR from the original public gateway

My understanding is that a realistic feature request here is to leverage a list of public gateways for content routing when loading data via local gateway takes a long time (more than n seconds), and fetch them as raw Block or CAR (in a way that still verifies every block).

This is a sensible feature request, but is not possible to do without Verifiable HTTP Gateway Responses (https://github.com/ipfs/in-web-browsers/issues/128).

tl;dr blocked until

lidel commented 2 years ago

Verifiable gateway responses are now possible: https://docs.ipfs.tech/reference/http/gateway/#trustless-verifiable-retrieval

Companion should be able to monitor URLs and log when a public gateway URL is requested, then detect when top-level document load takes more than a few seconds, and accelerate it by requesting blocks over HTTP from the original gateway.

This should be enabled by default, with opt-out toggle on Settings page.