neelvirdy
commented
1 year ago closing in favor of https://github.com/application-research/estuary-www/pull/92, can revisit saturn as a potential fallback gateway for when estuary gw is requested for content that is not pinned or has been offloaded, if dweb.link itself does not end up leveraging saturn
Opening the discussion of switching over the estuary-www preferred gateway to leverage Saturn.
Did some simple tests and found strn.pl to be anecdotally faster than dweb.link, but slower than api.estuary.tech/gw for new estuary uploads. I still need to test uploading new content via estuary and seeing how Saturn does in fetching it, particularly on first retrieval.
Another option is simply using api.estuary.tech/gw as the preferred gateway, which should be the most performant choice for first retrieval, but wouldn't get the caching benefits of Saturn and means the links could break if estuary was down.
Expanding on this – could we get Saturn to very quickly find estuary's gateways to improve speed when Saturn's L1 has a cache miss (i.e. first retrieval)? This would then make strn.pl a clear ideal choice for the preferred gateway on estuary-www, to me at least. One idea is having estuary gateways colocate a Saturn L1/L2 node, but this could result in the gateways caching a lot of files irrelevant to estuary users. If the level of customization allows it, we could potentially only cache estuary files on our node and preemptively write them to cache as part of an estuary upload.
Note: It seems still TBD how to handle clients that make requests directly to Saturn without payment: See "The client doesn’t pay directly for retrievals" on https://filecoin.io/blog/posts/retrieval-markets-rollup-h1-2022/. This is likely worth discussing with Saturn.