AshtonStephens commented 6 months ago

Design - Deposit API

This ticket holds the design of the Deposit API and the context around how it fits into sBTC-v1. It should specify the API calls that should be supported and justify them.

1. Summary

The Deposit API facilitates bitcoin deposits into sBTC from any type of wallet. It provides an alternative to using the op_return op code to carry layer 2 protocol data which requires non standard bitcoin transaction and places limitations on the the consumer wallet software that can be used to access the system.

Associated research:

2. Context & Purpose

To avoid the above problem, a two phase commit/reveal mechanism has been proposed that allows users to deposit from any bitcoin wallet. The user commits (sends bitcoin) to a taproot address via a standard bitcoin transaction. This transaction is then consolidated by the signers into the the sBTC wallet - in so doing the sBTC payload is revealed on the bitcoin network. The Deposit API serves as an intermediary between the user and the signers to enable the flow.

2.1 Payload Data

The taproot tree has the following properties;

Provably unspendable key path spend
two script paths
- reveal script - spendable by signers
- reclaim script - spendable by user (after elapsed time)

The Deposit API provides an end point for the user to generate the commitment address and stores the commitment data. The input required to generate the scripts is;

the stacks principal (account or contract) that's to be the recipient of the sBTC
the users public key (e.g. from connected wallet) for the reclaim script path
max fee - the maximum fee the user is willing to pay in bitcoin gas for the signers to reveal the data - see comments below and [5].

and the structure of the scripts is as follows;

reveal path

<sbtc_payload> DROP <signers_pubkey> CHECKSIG

where sbtc_payload, see;

0               8                                          159
|---------------|-------------------------------------------|
   max_fee             recipient_address

note the areas of ambiguity on whether max_fee is included in the payload or inferred from the deposit tx fee?

a suggestion that refines the recipient_address for the payload has been described previously;

3         4         5                25       26                         N <= 66
|---------|---------|-----------------|--------|-------------------------------|
principal  address       address       contract          contract name
type       version       hash          name length

note, bytes 0-3 are the magic and op code.

reclaim path

The specific reclaim script path is orthogonal to the overall sBTC design but could take following forms (note that OP_CSV is simpler to implement and decipher);

<lock-time> CHECKLOCKTIMEVERIFY DROP <user_pubkey> CHECKSIG
or
IF <lock-time> CHECKSEQUENCEVERIFY DROP reclaimPublicKey CHECKSIG ENDIF

the lock-time, be it relative or absolute, must be at least 1 full PoX cycle from when the deposit is confirmed.

2.2 Consolidation

Signer consolidation of the deposits is covered elsewhere. Question is a coordinator election needed here or can the signers wait a random time interval before polling and initiating the signing round ?

The Deposit API watches the bitcoin chain and removes user deposits from pending once it sees the reveal transaction.

3. Design

3.1 Proposed Component Design

A centralised RPC API approach is viable for sBTC-v1 other approaches are considered below.

3.1.1 Design Diagram

3.1.1.1 Signers poll for pending deposits

There can be many pending deposits - signers control the paging and the poll interval.

3.1.1.2 Signers poll for specific deposit

Signers poll by the bitcoin txid of the commit transaction.

3.1.1.3 User reclaim flow

User attempt to reclaim - the simplest version is the front end uses the public key form the users web wallet as signing and broadcasting the PSBT is then easily accomplished.

3.1.2 API Calls

Assuming the Signers poll for pending deposit transactions.

HTTP	API Call	Request Body	Params	Response	Success	Error
POST	/get-deposit-address	DepositRequest	-	DepositResponse	200	40x
GET	/reclaim-deposit			String	200	40x
GET	/get-pending-deposits		PendingDepositsParams	PendingDepositsResponse	200	40x
GET	/get-deposit		txid	PendingDeposit	200	40x

See appendix 1 for definition of api parameters.

/get-deposit-address

The user supplies the recipient stacks principal for the sBTC mint and the API encodes this data into taproot script path, spendable by the current sBTC signers, and returns the address generated for the user deposit.

The user also supplies a public key for the reclaim script path and optionally a value for max_fee.

Note: /get-deposit-address is a potential source of attack - see DOS attack.

/get-pending-deposits

Signers poll for deposits that have been paid by the user.

Returns empty list if none exist

/get-deposit

Signers request data for a specific deposit that has been paid by the user.

Return 404 if paid transaction is not found.

/reclaim-deposit

User attempts to reclaim deposit. API checks for the unspent TxO and for expiry of lock time. If ok generate an unsigned PSBT for the user to sign and broadcast.

Return 40x indicating conditions not met.

3.1.3 Considerations & Alternatives

commitment data is stored on chain - signers discover it via clarity - more complex and requires 2 transactions
commitment data is stored AND verified on stacks as per the discussion here; sBTC Research - OP_RETURN vs OP_DROP. In this scenario any bitcoin watcher can submit the transaction to the clarity contract, the contract verifies the transaction data and performs the mint - the signers no longer need to consolidate the UTxOs and can just spend them in the hand-off.
Question The current consolidation mechanism requires the signers to consolidate the UTxO and then call the .sbtc contract to accept the deposit. However an alternate suggestion was slated in [1] whereby the signers include OP_RETURN outputs in the consolidation transactions that allow stacks nodes to directly call the .sbtc contract, removing the need for a signer voting round - this is out of scope of sBTC-v1 ?

3.1.4 Security Considerations

The Deposit API is a centralised component required to enact the commit/reveal (OP_DROP) sBTC deposit flow.

To avoid exposing a public accessible IP on the signers it is recommended the signers poll the API as opposed to using web socket to push data to the signers.

Security concerns around using a centralised API can be further mitigated by deploying the OP_RETURN deposit flow as soon as possible as this side steps the discovery problem inherent in the commit reveal mechanism.

3.2 Areas of Ambiguity

Open questions;

how to handle max fee
precise spec of sbtc_payload
exact data needed to be passed back to signers

Closing Checklist

[ ] The design proposed in this issue is clearly documented in the description of this ticket.
[ ] Everyone necessary has reviewed the resolution and agrees with the proposal.
[ ] This ticket has or links all the information necessary to familiarize a contributor with the design decision, why it was made, and how it'll be included.

Appendix

Appendix 1: API Request / Response

API Parameters;

DepositRequest	Type	Description
originator	string	Stacks account initiating the deposit - may or may not be the recipient
recipient	string	Stacks account or contract principal
reclaimPublicKey	string	Public key for reclaim path
maxFee	string	Max fee the end user is prepared to pay for consolidation

DepositResponse	Type	Description
commitAddress	string	The address for the user deposit

PendingDepositsRequest	Type	Description
page	string	Page of deposits to return
limit	string	Number of deposits per page

PendingDepositsResponse	Type	Description
total	number	Total of pending deposits
pendingDeposits	Array	Tapscript data

PendingDeposit	Type	Description
address	string	commit address
script	string/Uint8Array
leaves	Array
tapInternalKey	string/Uint8Array
tapLeafScript	Array
tapMerkleRoot	string/Uint8Array
tweakedPubkey	string/Uint8Array

AshtonStephens commented 6 months ago

@radicleart this is fantastic

netrome commented 6 months ago

Re ambiguity, we have decided to have the max fee in the deposit UTXO as per https://github.com/Trust-Machines/sbtc-v1/issues/30

hstove commented 6 months ago

Overall looks good, but my main pushback is that the user needs to call /get-deposit-address, and then the deposit API polls BTC for transactions.

I think it's better to allow client-side generation of deposit addresses - it's just more flexible and less centralized. Then, instead of the deposit API "pulling", users should "push" when they've made a transaction.

AshtonStephens commented 6 months ago

Security concerns around using a centralised API can be further mitigated by deploying the OP_RETURN deposit flow as soon as possible as this side steps the discovery problem inherent in the commit reveal mechanism.

I don't think we plan to ever have OP_RETURN for sBTC

hstove commented 6 months ago

I'd also argue against /reclaim-deposit - maybe an API for metadata like "can this be reclaimed yet" makes sense, but otherwise everything else is probably better done client-side (and we can provide libraries for this)

AshtonStephens commented 6 months ago

Do we want to discuss the cloud infrastructure necessary to host this in this issue or a separate one? This could make sense as just the "interface" ticket.

radicleart commented 6 months ago

I'd also argue against /reclaim-deposit - maybe an API for metadata like "can this be reclaimed yet" makes sense, but otherwise everything else is probably better done client-side (and we can provide libraries for this)

Regenerating the taproot scripts depends on remembering max_fee. It's also possible that the sBTC wallet public key changes before the user attempts to reclaim.

radicleart commented 6 months ago

I think it's better to allow client-side generation of deposit addresses - it's just more flexible and less centralized. Then, instead of the deposit API "pulling", users should "push" when they've made a transaction.

This feels risky as it could lead to the users bitcoin getting stuck in the commit address. I can imagine myself forgetting where the client tab was after 2 phase factoring into Coinbase to pay the invoice, getting distracted by the kids, dinner being ready, someone closing my tabs etc.

Given the overlap with the above comment, I'd prefer to keep this flow and add a push end point for projects to deal with this responsibility client side if they chose to do so. Also to write the server side code to use a library (stacks.js etc) to generate the taproot scripts so this is easily replicated on the client ?

radicleart commented 6 months ago

Do we want to discuss the cloud infrastructure necessary to host this in this issue or a separate one? This could make sense as just the "interface" ticket.

sounds good.. the prototype is running on Linode infra but this will need to move to something with more bells and whistles !

AshtonStephens commented 6 months ago

I'm looking at https://github.com/stacks-network/sbtc-bridge-api and while I think we should reuse what we can, I'd like to push the idea that we might use AWS for the Deposit API with API Gateway as the interface, AWS Lambda for handling the requests, and potentially DynamoDB for any temporary data storage the service requires. I know that's a departure from what we did before, but using AWS will drastically simplify the code and reduce the number of concerns we need to handle within the API.

Would we be open to using AWS for this? I can create the AWS boilerplate so the implementer of this doesn't need to be trained in AWS ahead of time.

radicleart commented 6 months ago

I'm looking at https://github.com/stacks-network/sbtc-bridge-api and while I think we should reuse what we can..

FYI @AshtonStephens the repo sbtc-bridge-api is deprecated. I rebuilt the api as the revealer api - it is under my profile pending the project restarting. Better to use this as reference and scrap the stacks-network repo at some point.

AshtonStephens commented 6 months ago

Okay that makes sense. I do think though that our approach for sBTC-v1 should lean towards implementation speed and serviceability, which is where I think AWS would shine. We get a number of things for free (implementation wise):

Secret management
API throttling & API key management
Dynamic scaling
Metrics dashboards

THIS might be too ambitious, but I've come across an example repository where the API description is in smithy and the execution is written in a Rust lambda.

https://github.com/david-perez/smithy-rs-lambda-cdk

The main reasons to do this in Rust would be:

More people on the team can contribute to the API
It'll be easier / faster to audit
We can use the transaction libraries that @xoloki and @djordon are going to write

I made a simple website in AWS with a python lambda as the backend and an OpenAPI definition which could be used as boilerplate as well / instead:

https://github.com/AshtonStephens/simple-aws-website

I'm not saying we have to do it with AWS, but we should be deliberate in choosing a cloud provider and try to keep everything in the same place. That said, I really think we would shy away from any persistent hosting and do anything we can with server-less compute.

Resource dump:

xoloki commented 6 months ago

Totally agree that we should use Rust @AshtonStephens, for library reuse and team accessibility. As I said before, I've done this kind of thing before not only with the full-in AWS approach (they even sponsored my talk at GDC!), but also cloud-agnostic k8s.

There's a lot to be said for the lambda/serverless approach here, with a minimal bespoke attack surface. Deposit API shouldn't really be keeping a lot of state around anyway, it's basically just an entrypoint that does minimal processing and protects the signers. I do have an instant dislike of that smithy-rs-lambda-cdk repo for its use of gradle though XD

AshtonStephens commented 6 months ago

The revealer API that @radicleart linked here is using swagger to specify the API, smithy is just the api description language that Amazon made and has slightly better AWS hooks because of it but Smithy is also pretty limited last I checked so I'm against using it here (which means no gradle).

@radicleart we did explicitly say that node is fine for API implementation, now we're going back on it a bit. We should sit down and decide which method to go with.

@xoloki why would k8s be a better option than a rust lambda?

xoloki commented 6 months ago

The advantage to k8s is that you can move the app between clouds trivially, for resilience and disaster recovery.

Lambda will almost certainly be quicker to implement and have a small bespoke attack surface.

For v1 the balance likely favors lambda.

AshtonStephens commented 6 months ago

Once we choose a language I can create a boilerplate project that we can add the functionality to.

netrome commented 6 months ago

I'm skeptical about using AWS Lambda and DynamoDB for this service.

In my experience, depending on AWS-specific services makes integration testing and local development a hassle. Considering our aggressive timeline, I'd prefer keeping things simple and implement the API as a single binary that can easily be integrated in local development environments when we work on other components like, for example, the signer.

When it comes to databases, i'd prefer to either use Sqlite which we are already using extensively or PostgreSQL if we need something more advanced. In either case, DynamoDB has the same challenges of integrating it locally and I don't see any benefits over using an open alternative like postgres.

netrome commented 6 months ago

When we deploy the service, I have no strong reservation against using AWS specific services such as API Gateway, EKS or just a few EC2-instances. We can also use the AWS provided postgres. Although I'd still prefer to set up the deployment in k8s over using AWS-specific services.

netrome commented 6 months ago

Nit: The deposit request will require the full reclaim script, not only the reclaimPubKey.

xoloki commented 6 months ago

@netrome when you go all in on AWS you only do unit tests locally; all devs get personal clouds, and they run everything else there. k8s as we've discussed works fine with minikube etc.

I favor k8s in general, but there are a lot of gotchas when it comes to building out a scalable API layer that AWS will give us for free.

We should probably have a meeting about this topic specifically, or at least block off some time in an sBTC planning meeting.

netrome commented 6 months ago

The PendingDeposit has a lot of redundant information. The only information it should need are the two spending conditions and the txid and output index identifying the deposit UTXO.

Reference: https://github.com/Trust-Machines/sbtc-v1/issues/30

netrome commented 6 months ago

@netrome when you go all in on AWS you only do unit tests locally; all devs get personal clouds, and they run everything else there. k8s as we've discussed works fine with minikube etc.

I favor k8s in general, but there are a lot of gotchas when it comes to building out a scalable API layer that AWS will give us for free.

We should probably have a meeting about this topic specifically, or at least block off some time in an sBTC planning meeting.

Sure, in my experience that setup works well when you have simple applications that require massive scale. Our case is a complex application with loads of interaction points between services, with limited scale. Bitcoin essentially rate-limits how many deposit requests we can have per block, so we should not expect any load which a single node could not handle.

netrome commented 6 months ago

Also, I'm not opposed to having a scalable API layer in AWS for our deployment. That should still be possible if we implement the API as a simple service that is easy to run locally. If it's easy to run locally and stateless, it's trivial to scale it.

netrome commented 6 months ago

We should probably have a meeting about this topic specifically, or at least block off some time in an sBTC planning meeting.

Agreed. I'd love to further explore this in a synchronous dialogue.

radicleart commented 6 months ago

For context, the alpha (romeo) testnet sbtc web app uses the revealer-api is a node express app running in docker on Linode. The HLD above was based on this app as it has an impl of op_drop - generating the commit address on the server and storing the taproot data in mongo cloud.

FYI, the app has 4.5 tBTC in the peg wallet from ~ 1000 sBTC testnet txs. The tBTC needs to be reclaimed at some point.

It has most of whats needed for the deposit api and has a bunch of other calls that make the bridge app more efficient. It does require scaling infrastructure and k8 would be a logical choice as its already just a single docker container deployment.

I'm not personally attached to it and support rebuilding it in rust if this makes it more robust and aligned with the teams skill sets. That said keeping it as a single, slightly broader api and focusing on improving the existing typescript app may decrease TTM and increase head space.

I agree we should avoid vendor lock in and, in general, the empires takeover of the Internet.

xoloki commented 6 months ago

Sure, in my experience that setup works well when you have simple applications that require massive scale. Our case is a complex application with loads of interaction points between services, with limited scale. Bitcoin essentially rate-limits how many deposit requests we can have per block, so we should not expect any load which a single node could not handle.

To be clear, I don't want a scalable API layer because I think users are gonna be slamming us; I want something that can shrug off a DDOS attack. This is the value I see in an AWS rolled solution for the deposit API.

netrome commented 6 months ago

To be clear, I don't want a scalable API layer because I think users are gonna be slamming us; I want something that can shrug off a DDOS attack. This is the value I see in an AWS rolled solution for the deposit API.

Fair point, though DDOS protection is provided directly by AWS API gateway afaik, so we wouldn't need to go all in on Lambda for that. If we want auto scaling in addition to this, that shouldn't be a problem either.

My main point being if we start from a simple solution, we can add scalability and protection layers on top of the solution. I have nothing against leveraging AWS-specific services for this. As long as the foundation is simple. If we build on a complex foundation, it will be difficult for us to deliver fast and ensure the quality of our software.

netrome commented 6 months ago

For context, the alpha (romeo) testnet sbtc web app uses the revealer-api is a node express app running in docker on Linode. The HLD above was based on this app as it has an impl of op_drop - generating the commit address on the server and storing the taproot data in mongo cloud.

FYI, the app has 4.5 tBTC in the peg wallet from ~ 1000 sBTC testnet txs. The tBTC needs to be reclaimed at some point.

It has most of whats needed for the deposit api and has a bunch of other calls that make the bridge app more efficient. It does require scaling infrastructure and k8 would be a logical choice as its already just a single docker container deployment.

I'm not personally attached to it and support rebuilding it in rust if this makes it more robust and aligned with the teams skill sets. That said keeping it as a single, slightly broader api and focusing on improving the existing typescript app may decrease TTM and increase head space.

I agree we should avoid vendor lock in and, in general, the empires takeover of the Internet.

While I prefer to build APIs in Rust, I have nothing against a node express server. Especially if we already have a good foundation to build on top of. I'm happy to leave the actual language and framework choice to the owner of the component (which I guess would be you @radicleart). As long as we have something that is easy to depend on in other services (as in, simple to run locally which I feel like I've been nagging a lot about).

AshtonStephens commented 6 months ago

I think these are all very good points. The big thing I'd like to stay away from is a bespoke setup that only really @radicleart can work on, and that needs us to handle generic issues ourselves.

Local testing is definitely a weak point of using lambda and API gateway, but the AWS related components don't have that much that makes what we develop locked into using AWS. The core code knows nothing of AWS, it's just the interfaces for the binary that need to know.

The big thing I think we should stay away from if we can are multiple docker containers for this app alone. The fewer environments this API needs to consider the better for local testing and deployment.

Our case is a complex application with loads of interaction points between services, with limited scale.

I actually don't think this is true; the deposit API should just be an information relay and will take in user requests and get data from the signers and the Bitcoin node. It shouldn't feel too complicated.

netrome commented 6 months ago

I actually don't think this is true; the deposit API should just be an information relay and will take in user requests and get data from the signers and the Bitcoin node. It shouldn't feel too complicated.

I'm referring to the full sBTC system. When we build the Signer I would like to be able to iterate against a local version of the deposit API.

AshtonStephens commented 6 months ago

We've discussed and Typescript seems to make the most sense, and using EKS would likely make the most sense.

AshtonStephens commented 6 months ago

Now that I'm going to be handling the Deposit API I am actually going to go against the grain on this and use AWS lambda with rust, but I'll make sure that there's a way to run this locally - maybe that means there's some standard wrapper around the code.

I'll be able to spin up the lambda related architecture very quickly and have it be production ready from the start.

netrome commented 6 months ago

Now that I'm going to be handling the Deposit API I am actually going to go against the grain on this and use AWS lambda with rust, but I'll make sure that there's a way to run this locally - maybe that means there's some standard wrapper around the code.

I'll be able to spin up the lambda related architecture very quickly and have it be production ready from the start.

Have fun! As long as I can run it locally and use it in signer integration tests I'm happy.

AshtonStephens commented 6 months ago

Yeah, if that turns out not to be true if we use Lambda then I'll look for a different approach.

As in, local testing is a hard requirement so I'll forgo Lambda if local testing is infeasible.

AshtonStephens commented 6 months ago

Got Lambda working in a docker container - I'm hoping hooking it into an OpenAPI proxy will be easy but I'm running into issues with kong following this example:

https://thinkingeek.com/2023/12/24/testing-api-gateway-docker-lambdas/

AshtonStephens commented 6 months ago

I'm going to track my progress on running an AWS like environment locally here:

61

AshtonStephens commented 5 months ago

Closing this because we've got the information necessary from this conversation, further conversations will be in the LLD #50

stacks-network / sbtc