Make dealbot storage deals with a public good dataset

pooja commented 3 years ago

The dealbot wants to make deals with DataCap. The dealbot currently generates data to store for deal testing/reputation purposes. In a future iteration of dealbot, we should consider using a public good dataset to make dealbot storage deals with. One idea for how we can do this relatively straightforwardly in the future:

Take a dataset (doesn't have to be a massive dataset) and chunk it into 32GB chunks.
Store those chunks on the dealbot server
Dealbot makes deals round robining with those chunks. Many redundant copies of the full dataset will exist on the network with enough time. But also some deals will fail, so many iterations isn’t a bad idea.
Every 6mos or so, switch out the dataset

willscott commented 3 years ago

@pooja who's the right person to work with in order to ingest such a dataset for deals?

pooja commented 3 years ago

Hey just saw this. But @dkkapur would be the right person

On Sat, Jun 26 2021 at 11:44, Will < @.*** > wrote:

@pooja ( https://github.com/pooja ) who's the right person to work with in order to ingest such a dataset for deals?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( https://github.com/filecoin-project/dealbot/issues/246#issuecomment-869020445 ) , or unsubscribe ( https://github.com/notifications/unsubscribe-auth/ABLH2S743ZMSBCSVTAMM4WTTUXYWFANCNFSM47IFVRAA ).

dkkapur commented 2 years ago

Putting this as a P3 for now, we should discuss in the near future how we want to prioritize this against scaling dealbot up to get data from whole network for repsys.

filecoin-project / dealbot

Make dealbot storage deals with a public good dataset #246