Setting up our own SP is a necessary first step in offloading and decentralizing Estuary's data.
dog-fooding our own Storage Provider will allow us to feel the same pain as our current SPs
fix and react to changes faster
potentially provide our own certified software to SPs, steward our users’ data
serve to bootstrap an Estuary/Filecoin Retrieval Market
Once we have determined a root SP (or set of root SPs), one that should be owned by Estuary/Protocol Labs, we have to hard code all deals to flow into it, reducing the total deals by one. It is also very important that the retrieval deals are compensated back to Estuary to demonstrate the economics of the system (and help offset the price of running SP infra).
What scale do we want the SP node to be? Seagate makes an excellent storage device in a 2PB size, which could be a good starting point. If we are storing unsealed copies for everything then it'll be closer to ~0.8PiB available storage for deals. Need to land on an architecture where we can scale, if needed.
How will we seal? There are high capital costs associated with sealing hardware. As Sealing-As-A-Service / Remote Sealing becomes available, it is a much more flexible option as we can pay per-sector as required, and don't need to own/operate the hardware.
Setting up our own SP is a necessary first step in offloading and decentralizing Estuary's data.
Once we have determined a root SP (or set of root SPs), one that should be owned by Estuary/Protocol Labs, we have to hard code all deals to flow into it, reducing the total deals by one. It is also very important that the retrieval deals are compensated back to Estuary to demonstrate the economics of the system (and help offset the price of running SP infra).