oceanprotocol / market

🧜‍♀️ THE Data Market
https://market.oceanprotocol.com
Apache License 2.0
189 stars 292 forks source link

People can upload data easily (and we don't store it) #446

Closed trentmc closed 2 years ago

trentmc commented 4 years ago

Summary Provide publishers with an easy solution to drag & drop some files during publish flow and in background they are added to decentralized storage. But where we don't store it or pay for storage.

Motivation Status quo: publishers must a provide url for their data, which takes technical savvy. Removing this friction will help adoption.

In Ocean Commons, we had where people could "upload to IPFS" for a smoother flow. They would drag and drop the file, and it would get uploaded. However since IPFS does not do storage, we were actually doing the storage on Pacific nodes. This is not a good idea because then we inadvertently start hosting a ton of files for free.

Target outcome

Where to store

hebbianloop commented 2 years ago

Our team is working on this! As mentioned in our recent issue/PR, overhead is still a challenge. There may be some ways to decrease overhead but will require a bit of engineering + experimentation https://github.com/oceanprotocol/market/issues/1483

https://port.oceanprotocol.com/t/project-coral-core-tech-improvement-desci-oceandao-berlin-hackathon-round-17/1824

trentmc commented 2 years ago

our recent issue/PR

To be clear, that's an issue. And it's fine as an issue right now, given that there is discussion on the specs. A PR will be needed at some point though.

kremalicious commented 2 years ago

As a reminder cause it's mentioned in this issue's description as a goal. This is how the file form field looked in Commons:

Screen Shot 2022-06-06 at 12 49 43

"From URL" revealed the URL text input, "Add to IPFS" opened up a drag & drop field, dropping a file was sending this to our own IPFS node's API and we pinned it there. App then receives back the new IPFS URL and uses that as the final file URL. No centralized 3rd party needed at all. We don't run our own IPFS node anymore, but nice solutions like https://web3.storage came along anyway

mihaisc commented 2 years ago

We have other issues that handle this in a more direct way