Rework current pinning & storage strategy

[yangwao]

Hence there were unknown changes or at least on Basilisk we've noticed that after fresh minting things are not working. Thus time to revisit the initial implementation of cf workers and how we have it in the current state.

Currently, the user pins stuff on Estuary, which we've found acting suboptimal as it's being stored on Filecoin (archive nodes(?) and retrieving through pinata gateway takes a long time (peering issues?). (Something has changed outside of our control, we are investigating, it till we are looking to mitigate it and make a more resilient and antifragile scenario)

Future proposed changes are

• [x] #3358 temp hotfix
• [ ] Pinning metadata on Pinata (as we are blazingly fast here for end-user experience) and rest pinning could be kept on Estuary -> We are pinning rn on NFT.storage and seems temporary hotfix is okay
• [ ] At minting, we should push media to CF images (might not happen during BSX mints?)
• [ ] At the new implementation, we would doesn't need durable objects at all and fetch images by IPFS hash from CF images to accelerate end-user experience while they browsing collections/explorer
• [x] https://github.com/kodadot/nft-gallery/issues/3745
• [x] https://github.com/kodadot/nft-gallery/issues/3296

Long-term goal

Currently, we are fairly doing close to 1TB on Pinata Gateway monthly as transfers for users and this would be an interesting way to offload as public goods for storage providers, like those around the Filecoin provider's circle

Suggested would be to recycle code from

• https://github.com/kodadot/nft-gallery/blob/main/src-functions/pinJson.js
• https://github.com/kodadot/workers/blob/master/pinning/src/lib.rs#L84

To make it work with Pinata directly from workers to pin JSON metadata there

Check the links below for more context Let's comment below if you want to thrill on this issue

Ref

• https://github.com/kodadot/packages/issues/17
• https://github.com/kodadot/workers/issues/7
• https://github.com/kodadot/workers/issues/8
• https://github.com/kodadot/nft-gallery/issues/3253
• https://github.com/kodadot/bao/issues/10

[yangwao]

Seems I've found a quick alt, we can give it shot with nft.storage

https://nft.storage/api-docs/

I've uploaded pic through their interface and it was immediately available through our Pinata Gw https://kodadot.mypinata.cloud/ipfs/bafkreie4kobhhrqco2n7s4jwx7vhgciw5kiojluvvijoifazrsh5ipxw4m

https://filecoinproject.slack.com/archives/C016APFREQK/p1657104635066799

I've asked difference between Estuary <> NFT.Storage <> web3.storage

[yangwao]

we are fairly doing close to 1TB on Pinata Gateway monthly

well, it was over 2TB recently - Total Requests: 4049068 - Bandwidth: 1.82 TB and seems on Carnival (biggest plan they offer in public), cap is 10M req where we already peaking at 2T watermark around 4M req

[yangwao]

Hence we have

• 4587

I'll review this one, but there is not much to do. (?)

[yangwao]

Happy solved