Implement Auto Top Up Worker

[baktun14]

Objective

Implement a worker CLI handler that automatically adds funds (top-ups) to Akash Network deployments when they are low on balance. This ensures deployments continue to run without requiring users to manually monitor and replenish funds, improving the user experience.

Scope

• The worker should interact with the deployment grants from the chain and validate wallet balances to ensure sufficient funds.
• Focus on the backend (server-side) implementation, integrating with the required services.
• The client-side will be developed after this worker is implemented and is out of scope for this task.

Details

General Workflow

1. Target Owners Detection

   • For custodial wallets retrieve the deployment deposit grants given to the master wallet from the chain.
   • For managed wallets retrieve all the wallets from database.
   • For all of the above extract wallets addresses as deployments' owners.

2. Draining Deployments Detection (per every owner)

   • Retrieve all deployments with grants that are about to drain (use calculations provided below).

3. Top-up Action (per every deployment)

   • Calculate the required top-up amount (use calculations provided below).
   • For custodial wallets verify owner has sufficient balance to top up.
   • For managed wallets verify owner is granted sufficient funds to top up.
   • For custodial wallets send a transaction to the dedicate master wallet of the relevant denomination to top-up the deployment with the owner as the granter.
   • For managed wallets send a transaction to top-up the deployment to the same wallet with the master wallet as the granter.

Acceptance Criteria

• The cli command is a part of the new "deployment" module in the API.
• The handler utilises async techniques (e.g., promise pooling) to handle multiple deployments and wallets efficiently.
• Proper separation of concerns is followed. New http services, repositories, db schemas are added where applicable (suggested structure is on the design chart).
• Proper error handling for failed transactions and retries if applicable.
• Logging to provide sufficient level of visibility of actions taken or failed.
• Functional tests coverage.
• Worker runner in GCP is configured.

Calculations

  // of the above is estimated block height of closure
  Math.ceil(deployment.lastWithdrawalHeight || lease.createHeight) + deployment.balance / blockRate

• blockRate = sum of deployment leases' prices.

The above can be used to calculate the time when a deployment is expected to close. To achieve that average block time is needed which can be found in apps/api constants.ts or blocksService.ts.

Important to include job run interval into calculations to ensure not missing the time of closure.

To calculate amount to top up we need to get a number of blocks per time unit which is duration / average block time and multiply that by blockRate.

Design Chart For Reference

Link

Initial task version below

• When creating a lease transaction, add a new transaction that authz our master wallet to deposit on their behalf for the active deployments
• Implement a ui in the deployment detail to configure the top ups
  • Threshold to top up
  • Duration of the top up based on the cost of the lease
• Implement a scheduled job that goes through all the auto top ups and deposit on their behalf
• Using the indexer, based on the lease expectedClosedHeight + threshold + cost of lease, create a top up event
• Add new table to create a job with type (deployment escrow top up in this case)

[baktun14]

Need #398

[ygrishajev]

Started compiling a design chart

[ygrishajev]

Need #398

just for transparency, after a discussion we agreed there is a way to implement it without dependency on auth

[ygrishajev]

Right now there's a limitation of DepositDeploymentAuthorization that it only allows for one denom meaning we cant have same granter/grantee for both uakt and usdc grants. So there's an update to requirements:

Implement 2 new master wallets specifically for Auto Top Up. 1 wallet should serve for uakt top ups, and another for usdc grants.