feat: Egress Traffic Tracking + Stripe Billing Meters

[fforbeck]

Egress Traffic Tracking and Stripe Billing Meters API Integration

This PR introduces an asynchronous solution for tracking egress traffic in the Freeway Gateway and reporting events to Stripe’s Billing Meters API.

RFC Reference

Storacha Network is implementing a scalable, automated mechanism for tracking egress traffic and updating Stripe’s API with relevant data to ensure accurate customer billing. This RFC outlines the proposed approaches and their trade-offs.

Implementation

The selected approach for this implementation is Alternative 3:

• Freeway Worker invoking usage/record, SQS, Lambda with Stripe Integration

Flow

1. To avoid blocking, the Freeway Gateway asynchronously invokes the usage/record capability using ctx.waitUntil.
2. A new handler added to w3infra/upload-api/stores/usage.js receives the events from the Kinesis stream and places the egress data into an Egress Traffic SQS queue.
3. A new Lambda function consumes the events from the SQS queue and publishes the egress data to the Stripe Billing Meters API.

graph TD
    CF[Freeway Gateway] --> w3infra[w3infra/upload-api]
    w3infra --> Kinesis[Kinesis Stream]
    Kinesis --> UsageRecord[Usage Record Handler]
    UsageRecord --> SQS[Egress Traffic Queue]
    SQS --> Lambda[Lambda Function]
    Lambda --> DynamoDB[Egress Traffic DynamoDB Table]
    Lambda --> Stripe[Stripe Billing Meters API]
    UsageRecord --> Logs[S3 Bucket /Receipts]

Key Advantages

• Asynchronous execution: ctx.waitUntil ensures the Freeway Worker won’t block, allowing seamless egress traffic handling.
• Decoupled architecture: By using SQS and Lambda, the architecture is decoupled and includes a buffer before publishing events to Stripe.
• Signed receipts: Each egress event generates a signed receipt, providing auditability and traceability. The receipts are stored in the general bucket in S3 where we already store other receipts.

Summary of Changes

• Introduces the usage/record capability handler in w3infra/upload-api/stores/usage.js to handle egress data from the Kinesis stream and place them into the SQS queue.
• Implements an event-driven architecture using SQS and Lambda to process egress events asynchronously and publish to Stripe.
• An integration test that publishes fake egress traffic events on Stripe’s Test API.

Next Steps

• Deployment (Staging)
• Validate the solution’s performance and scalability.
• Monitor for potential rate limit issues when publishing to Stripe.

[seed-deploy[bot]]

View stack outputs

- **pr430-w3infra-BillingDbStack** Name | Value -- | -- customerTableName | pr430-w3infra-customer egressTrafficTableName | pr430-w3infra-egress-traffic spaceDiffTableName | pr430-w3infra-space-diff spaceSnapshotTableName | pr430-w3infra-space-snapshot usageTable | pr430-w3infra-usage - **pr430-w3infra-BillingStack** Name | Value -- | -- ApiEndpoint | https://0z5d2c8dnb.execute-api.us-east-2.amazonaws.com billingCronHandlerURL | https://vty73rrprd5yctut7yy5xdlira0htnuo.lambda-url.us-east-2.on.aws/ CustomDomain | https://pr430.billing.web3.storage EgressTrafficQueueURL | https://sqs.us-east-2.amazonaws.com/505595374361/pr430-w3infra-egress-traffic-queue - **pr430-w3infra-CarparkStack** Name | Value -- | -- BucketName | carpark-pr430-0 Region | us-east-2 - **pr430-w3infra-PSAStack** Name | Value -- | -- downloadFunctionURL | https://gzfzfywxxobmmmazt4zaunda2u0dyuta.lambda-url.us-east-2.on.aws/ hashFunctionURL | https://lntduzzoh5afpo5ulbraoj7pci0rvsdo.lambda-url.us-east-2.on.aws/ - **pr430-w3infra-RoundaboutStack** Name | Value -- | -- ApiEndpoint | https://8gdomj05p9.execute-api.us-east-2.amazonaws.com CustomDomain | https://pr430.roundabout.web3.storage - **pr430-w3infra-UcanInvocationStack** Name | Value -- | -- invocationBucketName | invocation-store-pr430-0 taskBucketName | task-store-pr430-0 workflowBucketName | workflow-store-pr430-0 - **pr430-w3infra-UploadApiStack** Name | Value -- | -- ApiEndpoints | ["https://3qb2qziqu6.execute-api.us-east-2.amazonaws.com","https://hkbr5dg2h0.execute-api.us-east-2.amazonaws.com"] CustomDomains | ["https://pr430.up.web3.storage","https://pr430.up.storacha.network"] - **pr430-w3infra-BusStack** - **pr430-w3infra-FilecoinStack** - **pr430-w3infra-IndexerStack** - **pr430-w3infra-ReplicatorStack** - **pr430-w3infra-UcanFirehoseStack** - **pr430-w3infra-UploadDbStack**

[fforbeck]

Generally LGTM with one big comment:

I don't think we will have the customer DID when we call usage/record in freeway. @Peeja can you confirm that I'm right on that? Or maybe @travis has more context.

I'm sorry I didn't catch this earlier. You should be able to look it up via the subscription for the space.

Thanks for the review. No problem. That's right @hannahhoward , we won't have the Customer DID in the Freeway project, but we will have the Space DID, and this is what we will pass in the invocation. The w3up project was updated to derive the Customer DID from the Space+Provider: https://github.com/storacha/w3up/blob/main/packages/upload-api/src/usage/record.js#L18, and then it pushes the event to the queue with the Customer DID expected in the w3infra.

[hannahhoward]

Perfect block removed