search

nftstorage / nftstorage.link

🪐 NFT.Storage Gateway, the IPFS gateway for NFT.Storage is not "another gateway", but a caching layer for NFTs that sits on top of existing IPFS public gateways. ***Notice: Uploads have been decommissioned.**** Learn more and find a new hot storage provider for uploading new assets: nft.storage/nft-storage-classic
https://nft.storage/nft-storage-classic
Other
47 stars 10 forks source link

Gateway does not resolve all CID encodings #111

Closed sfrdmn closed 1 month ago

sfrdmn commented 2 years ago

Expected

When I visit https://nftstorage.link/ipfs/f0170122041cd1126759340060ea717da677ccf97619097af67a60e25fa526452d47e2e34, that is, an "f" base CID, the resource should resolve correctly

Actual

Server responds with 400 invalid CID: f0170122041cd1126759340060ea717da677ccf97619097af67a60e25fa526452d47e2e34: To parse non base32 or base58btc encoded CID multibase decoder must be provided in body

Notes

Taking the equivalent base32 encoding from https://cid.ipfs.io/#f0170122041cd1126759340060ea717da677ccf97619097af67a60e25fa526452d47e2e34, I can resolve the resource on the same gateway (bafybeicbzuism5mtiada5jyx3jtxzt4xmgijpl3huyhcl6ssmrjni7rogq)

dchoi27 commented 2 years ago

@vasco-santos mind taking a look?

vasco-santos commented 2 years ago

hey @sfrdmn

Thanks for reaching out. We currently use the base encodings from multiformats https://github.com/multiformats/js-multiformats/blob/v9.6.5/src/cid.js#L382-L404 . Any reason for not use base32 only?

We should be able to use all the base encodes from https://github.com/multiformats/js-multiformats/blob/v9.6.5/src/basics.js#L20 but we need to parse them outside CID validation before https://github.com/nftstorage/nftstorage.link/blob/main/packages/api/src/utils/cid.js#L32 and provide the base function to use.

Would you be interested in submitting a PR for this?

sfrdmn commented 2 years ago

Hi @vasco-santos

Sorry for the late reply. For me the reason is being able to derive IPFS URLs from NFT token ids themselves. Token ids are represented in 256 bits, which is not enough to hold all of the information in an IPFS URL. They can hold a 256 bit hash, though. However, in the case of base32 URLs, to build a a valid URL from a hash I cannot simply use a string prefix to form the rest of the multiaddr, I have to actually implement some sort of base32 encoding on chain to get the URL. In the case of base16 addresses, string concatenation is fine, basically because the "words" always fall evenly before the content hash portion of the address (as opposed to a 32 bit word spanning the content hash and other components of the address). But if I have a base16 URL and it doesn't actually resolve on any pinning services, that's also not great

Also it's just more powerful in general of course

vasco-santos commented 2 years ago

148 fixes this for subdomains. We still need to support path