search

nftstorage / nftstorage.link

🪐 NFT.Storage Gateway, the IPFS gateway for NFT.Storage is not "another gateway", but a caching layer for NFTs that sits on top of existing IPFS public gateways. ***Notice: Uploads have been decommissioned.**** Learn more and find a new hot storage provider for uploading new assets: nft.storage/nft-storage-classic
https://nft.storage/nft-storage-classic
Other
46 stars 10 forks source link

CAR format support 🌐✨🚘 #147

Closed olizilla closed 3 weeks ago

olizilla commented 2 years ago

Being able to fetch content as a CAR from nftstorage.link would be rad. It'd allow folks to fetch and verify content themselves, with the nftstorage.link speed boost.

We need it for the web3.storage client.get method. We could hit the nftstorage.link endpoint directly from that rather than via the api worker. We're getting reports of larger files failing to fetch via GET /cid/:cid which is currently just a proxy for ipfs.io. e.g https://github.com/web3-storage/web3.storage/issues/840

The implementation should just use the new /ipfs/:cid?format=car search param that landed in go-ipfs v0.13. If other gateways take a while to adopt this we should implement gets as hitting ipfs.io only, so at least we start to benefit from the cdn / caching behavioiur for them in the meantime.

olizilla commented 1 year ago

right now it's often more reliable to do a 

dagula get <cid I got from w3>

than it is to do a

w3 get <cid I got from w3>

because the current implementation of web3.storage.get() is a proxy for the dweb.link gateway. The content may not be on those machines, and they get very busy with public use.

what if we add /ipfs/:cid?format=car support by only racing upstreams that support CARs along with adding a new race participant as a dagula client that fetches the blocks from elastic-ipfs...

Jorropo commented 1 year ago

I just want to link the relevent spec as well as mention that Accept: application/vnd.ipld.car is the way we expect applications to use this feature (this is because thx to HTTP2 this allows to only send the header once). https://github.com/ipfs/specs/blob/main/http-gateways/TRUSTLESS_GATEWAY.md#accept-request-header