search

livepeer / go-tools

Utility packages used across Livepeer Go repositories.
0 stars 2 forks source link

Add IPFS support #3

Open cyberj0g opened 2 years ago

cyberj0g commented 2 years ago

Implement same interface as for S3 and other storage providers.

Suggested cloud provider: pinata.cloud

@victorges @red-0ne I believe you already worked on IPFS support, could you share findings or suggestions, which may be helpful?

@thomshutt tagging for visibility

red-0ne commented 2 years ago

It seems that we can either have it by implementing an object store (https://github.com/livepeer/go-tools) for pinata.cloud or use a custom impl. Victor has a pinata client on the task runner codebase https://github.com/livepeer/task-runner/blob/1ff546da1d51496d276cf84765b3abcbc387c1fc/clients/ipfs.go#L36 that could help.

victorges commented 2 years ago

Added some comments in the other issue on catalyst: https://github.com/livepeer/catalyst/issues/154

The suggestion I have is that you could reuse some code from the existing Pinata client when creating the OS driver. It's currently here: https://github.com/livepeer/task-runner/blob/1ff546da1d51496d276cf84765b3abcbc387c1fc/clients/ipfs.go#L36

Might help you skip some of their idiosyncrasies like not using CIDv1 by default and dealing with their multi-part requests format.

Also don't worry about implementing the drivers.Driver interface extensively. Some methods may not be possible on Pinata/IPFS and it's ok to just panic or return an errors.New("not supported") or something like that. On Arweave for example we won't be able to delete things. On Pinata we can delete, but the List method makes much less sense. I've only created it to use in some clean-up scripts, but not really in production. So I wouldn't bother too much in translating that into the Driver interface (but if you do, that's a nice thing to have anyway).

victorges commented 2 years ago

Oh for some reason Github didn't show @red-0ne's comment before mine so I repeated some things =P