WHYPFS dedicated gateway provisioning and subscription

[alvin-reyes]

Idea/Proposal: Dedicated Estuary Gateway provisioning and subscription

Contributors	@alvin-reyes
Status	Draft
Revision

Proposal

NOTE: This is a draft and is not finalize yet. We'll have to polish it until we all agreed on the approach.

We need to allow users to avail their own dedicated gateway so they can directly interact with for their contents.

1 - user needs to subscribe to a gateway. We will need to ask for the parameters

• Name (for domain name)
• Storage Size
• Payment based on storage size + service

2 - We need to develop a wizard like page to get gateway information.

• user clicks on "request" dedicated gateway". This launches a wizard step by step to get information from the user.
• Information - gateway name, storage size, payment method
• payment method - if user wants FIL, we use the FEVM deposit contract, otherwise - STRIPE or PAYPAL

3 - we need to develop a page for each user to navigate and manage their gateway(s).

• List view of all created gateways
• Dedicated page for a specific gateway

4 - middleware code

• The user needs to have an API key to access each gateways.
• The user needs to upload their files on their dedicated gateway and we will need to have an authentication to only serve contents from the user uploaded on the specific gateway only
• License file to be generated for the user with all the META of the subscription.

5 - backend

• dockerized components
• ansible scripts to provision the gateway. We need to provision the gateway with the defined resources on the docker-compose yaml file. The yaml includes the META information of the subscription, resources, storage, domain name, certificate generation, server and the WHYPFS gateway.

[jimmylee]

1 - user needs to subscribe to a gateway. We will need to ask for the parameters

• Yes on Name.
• We keep storage size fixed for now. We don't allow for variable spend, we don't know our own margin yet.

2 - We need to develop a wizard like page to get gateway information.

• Payment flow for this makes sense
• Use FEVM deposit contract into a Filecoin address that we own
• Double book accounting and ensuring that we know the current Filecoin to USD price.
• If we build a Stripe flow (I pick Stripe, we probably can end the debate here), we need to do a order for Filecoin and then deposit that Filecoin into our Filecoin address. I want to make sure that USD still triggers demand size usage for Filecoin.

3 - we need to develop a page for each user to navigate and manage their gateway(s).

• List all gateways the user has created (but not gateways that belong to other users)
• Dedicate page for the gateway with uptime statistics, at least on par with BetterUptime

4 - middleware code

• Yes to API key to access gateways
• What do you mean users need to upload files on their dedicated gateway, shouldn't we handle this for them?
• Cool idea with the license, we'll also need an balance page for this.

5 - backend

• Yes to docker kind of a given for this
• Ansible scripts are also a given here
• What is the best certification generation tool? lets encrypt?

Thanks for taking the initiative on this.

[10d9e]

@alvin-reyes Clarifying question:

we will need to have an authentication to only serve contents from the user uploaded on the specific gateway only

This might be tough if we are serving everything through bitswap via WhyPFS, right?

[alvin-reyes]

I was thinking we can create an authentication on the DAG service of the whypfs-core.

https://github.com/application-research/whypfs-core/blob/main/whypfs.go#L694

func (p *Node) GetFile(ctx context.Context, c cid.Cid) (ufsio.ReadSeekCloser, error) {

        **// AUTHENTICATE first either here or the CALLER of this function.**

    n, err := p.Get(ctx, c)
    if err == nil {
        return ufsio.NewDagReader(ctx, n, p.DAGService)
    }
    if err != nil {
        return nil, err
    }
    return ufsio.NewDagReader(ctx, n, p)
}

We can create a custom node that's peered with other IPFS node AND authenticate the CIDs before it actually PULL or PUSH the file. The authentication protect the methods to get and add file (on the code above).

If someone else pulls the data, a different node or gateway, I think we can live with this since this just increases the replication and retrievability rate of the CID - it'll fits with our value proposition of reliable retrieval of CID.