Output Unification

[guidanoli]

📚 Context

Currently, the off-chain machine is able to generate two types of outputs: vouchers and notices. On-chain, these outputs can be validated by anyone with the help of validity proofs.

The proof consists on reconstructing the epoch hash and comparing it with the epoch hash that was claimed by the DApp's consensus. If they are equal, then we can be sure the output is valid.

Currently, this epoch hash is composed by the following components.

• The machine Merkle root hash
• The voucher Merkle root hash
• The notice Merkle root hash

As you can see, vouchers and notices each have their own separate Merkle tree. This helps to avoid misinterpreting notices as vouchers and vice versa.

There are, however, two main issues with this design. First, it unnecessarily complicates the on-chain and off-chain code. Second, it hinders the implementation of new output types, since we would have to add a new Merkle tree, which would result in a breaking change in the validation procedure.

✔️ Solution

And so, we noticed that things would be much simpler if we had a single tree for all verifiable outputs. To differentiate between output types, we could add a prefix to each output. For example, 0x00 for vouchers and 0x01 for notices. As a result, the epoch hash would only be composed of two components:

• The machine Merkle root hash
• The output Merkle root hash

[tuler]

Can you add more context on why do we need more output types? Can you give examples?

[guidanoli]

Can you add more context on why do we need more output types? Can you give examples?

Of course. Take @fvbizzo's effort in implementing Complex Vouchers as an example. One cannot currently encode a timed voucher, or a voucher dependency, or a permissioned voucher. This is why we'd need to create new output types for these kinds of use cases. In reality, most—if not all—of these complex vouchers could be implemented with only one new output type: a voucher that could do a DELEGATECALL instead of a regular CALL opcode. Still, we cannot execute an arbitrary DELEGATECALL with the vouchers we have today, and thus the need to create new output types.

[miltonjonat]

Could you give practical use case examples for these suggested complex vouchers? For instance, I'm not sure in which application we would need a timed voucher, or if we have had any real demand for these. I also don't even understand what a permissioned voucher is ;)

[guidanoli]

@miltonjonat Sorry, I meant to say targeted vouchers! Let me try to give some examples of use cases for each of the vouchers proposed in the Complex Vouchers RFP.

• Ordered vouchers:
  • Indirect asset transfers: In order to interact with contracts that operate on the sender's tokens, the sender must first tell the token contract that they approve the operation. This is no different for DApp contracts. For example, say a DApp wants to transfer some amount of ERC-20 tokens to another DApp. They must first allow the ERC-20 portal to transfer such an amount, and then call the depositERC20Tokens function on the portal^2.
• Paid vouchers:
  • Calling payable functions: Some functions contain the payable keyword, meaning that they can be called while passing some amount of Ether. One example is the deposit() function of the Wrapped ETH token contract^1. It converts the Ether sent along into Wrapped ETH tokens and deposits them into the sender's balance. This may be useful for some DApps that want to provide liquidity in WETH.
• Re-executable vouchers:
  • Unlimited approvals: Some token contracts allow users to call approve() with some magical number, such as type(uint256).max, so that the allowance never gets consumed by future transfers. This, however, is not the case for all token contracts. In fact, this behavior is not even specified by EIP-20. To take these cases into consideration, the DApp may either emit approve() vouchers for every transfer. With re-executable vouchers, however, the DApp would only need to yield one approve() voucher.
• Targeted vouchers:
  • Controlled asset withdrawals: Users might not want to withdraw their assets ASAP. To give users control over when their assets are withdrawn from the DApp contract, we may want to restrict who can execute certain vouchers.
• Expirable vouchers:
  • Limited time asset withdrawals: Suppose you are the developer of a decentralized MMORPG DApp, and you've just launched the main net release party to welcome the new users. To show your gratitude towards the newcomers, you also decide to gift everyone that subscribes in the first month with an exclusive "Main Net Release Hat" NFT that they can wear in-game. This NFT can be retrieved through a voucher that will expire one month after the main net release date. On the second month, the NFT will only be available to those present on the first month or for those who purchased from them. You, the DApp developer, may even decide to emit a voucher that will burn any NFTs left in the stock.
• Future vouchers:
  • Scheduled asset withdrawals: Suppose again you are the developer of a decentralized MMORPG DApp, and that you'd like to award users with a Cake NFT on their birthday (which may be informed upon subscription). The problem is that your DApp is subscribed to a consensus that publishes claims once a week, and you'd like users to withdraw their Cake NFT on the exact day of their birthday. To solve this problem, your DApp will emit a voucher that transfers a Cake NFT to every user on their birthday. This voucher will be emitted with generous anticipation, so that the consensus has more than enough time to submit the claim containing the voucher.
• Atomic vouchers:
  • Indirect asset transfers: Same example as the one for Ordered Vouchers. The catch is that you can execute both vouchers in the same executeVoucher() call. This would be more gas efficient, because you wouldn't need to query the DApp's consensus for the epoch hash twice, and would facilitate the execution of multiple calls by an EOA. Additionally, you wouldn't need to link transfers as Ordered Vouchers, since the approval given to the recipient would be consumed by the subsequent transfer immediately.

[pedroargento]

Time based vouchers also have a bunch off applications in finance. For example, we could have a forward operation that allows me to buy X amount of a token in a future date for a certain price x. The dapp can generate two vouchers at the moment of the deal that can only be executed at the forward contract expiration date: one transfer of x eth from the buyer to the seller and a transfer of X tokens to the buyer.

Complex vouchers are being developed in response to RFP#003.

[guidanoli]

An alternative encoding for outputs

Currently, notices and vouchers are encoded with abi.encode.

abi.encode(notice)
abi.encode(destination, payload)

In order to unify the outputs in the same tree, we need to make their encodings disjoint (to avoid ambiguity and make it easier to detect the type of outputs). In this issue, I've proposed to prepend the output with a header like 0x00 or 0x01.

bytes1 constant NOTICE = bytes1(0x00);
bytes1 constant VOUCHER = bytes1(0x01);
// [...]
abi.encodePacked(NOTICE, notice)
abi.encodePacked(VOUCHER, destination, payload)

The problem with this solution is that if we create output types that have more than one arbitrary-length field, we'll have to use abi.encode instead of abi.encodePacked, or a conjunction of both. If we only use abi.encode, then either the header will have an unnecessary padding of 32 bytes, which will make it different from the encoding of the other outputs. If we use both, then we'll have an extra cost of double encoding. Another issue is that the choice of headers is completely arbitrary. With every new output type, we'll have to give it a new arbitrary number.

To solve these issues, I've come up with a better solution: to use Solidity's function call encoding. This is how the encoding of notices and vouchers would look:

abi.encodeWithSignature("Notice(bytes)", notice)
abi.encodeWithSignature("Voucher(address,bytes)", destination, payload)

With this encoding, we can use the output name as the function name, and the output fields as the function arguments. This removes the burden of having to come up with arbitrary numbers for output types, and uses the strict encoding of arguments out-of-the-box (like we do now). Besides, Solidiy already supports a nicer syntax for this encoding.

[ZzzzHui]

Very nice idea. abi.encodeWithSignature results in 32-4=28 less bytes than only using abi.encode with a header

[guidanoli]

Now, from the perspective of the machine, outputs can be thought of as arbitrary blobs. You will be able to tell them apart by the first 4 bytes, which correspond to the function selector of the function call. How this will be reflected on the HTTP API is still under discussion, but some options are:

• The generic approach: outputs are blobs.

  • Pros: Extensible.
  • Cons: User must encode outputs.

• The typed approach: outputs are either notices or vouchers or [...]

  • Pros: User-friendly.
  • Cons: Forbids users to emit new types of outputs.

• The mixed approach: outputs are blobs, which can represent notices or vouchers or [...]

  • Pros: User-friendly and extensible.
  • Cons: User may need to encode outputs, if they are not yet covered by the API.

[miltonjonat]

Given that this changes the public API that devs use, maybe it would be nice to consider inputs from the Prototyping and DevAd units. @cf-cartesi @gbarros would you like to comment?

[guidanoli]

Encoding and decoding outputs in Solidity

So, recently I discovered a nice idiomatic, type-safe way to encode function calls (which we'll be doing a lot). Here's the step-by-step process. I'll link to files from the output unification branch for reference.

1. Define the function in an interface. The nice thing about this approach is that we can use this definition elsewhere, either when encoding or decoding these function calls. You can also easily add documentation in NatSpec format, which can be later used by solidity-docgen to generate documentation in Markdown. Furthermore, off-chain code will be able to easily encode/decode data through the contract ABIs and their language bindings. https://github.com/cartesi/rollups-contracts/blob/8c0a23ec835d9b58191a9f60f979526f1410776e/onchain/rollups/contracts/common/Outputs.sol#L9-L20
2. Encode the function call data using abi.encodeCall, passing the function pointer as the first argument. After that, the function arguments are passed as a single tuple. And good news: this method is completely type-safe! That means that the compiler will raise an error if you try to encode a function call with the wrong arguments. Language bindings in typed languages will most likely perform these type checks as well. https://github.com/cartesi/rollups-contracts/blob/8c0a23ec835d9b58191a9f60f979526f1410776e/onchain/rollups/test/foundry/dapp/CartesiDApp.t.sol#L728-L732
3. Decode the function call. First, you need to check if the byte array starts with the right selector. For this, I wrote a library called LibCalldata, which has a function called trimSelector. https://github.com/cartesi/rollups-contracts/blob/8c0a23ec835d9b58191a9f60f979526f1410776e/onchain/rollups/contracts/library/LibCalldata.sol#L14-L24
4. With the selector trimmed, you can decode the function arguments with abi.decode. Here, you'll have to specify the argument types, unfortunately. https://github.com/cartesi/rollups-contracts/blob/8c0a23ec835d9b58191a9f60f979526f1410776e/onchain/rollups/contracts/dapp/CartesiDApp.sol#L140-L143

[ZzzzHui]

abi.encodeCall was introduced in 0.8.11 and bug fixed in 0.8.13 (ref), so we probably need pragma solidity ^0.8.13;

[gligneul]

I created an issue in rollups-node repository to track the changes on our side: https://github.com/cartesi/rollups-node/issues/103

[mpolitzer]

The machine-sdk side changes are being tracked here: https://github.com/cartesi/machine-emulator/issues/137

[guidanoli]

OpenZeppelin v5 also migrated from abi.encodeWithSelector and abi.encodeWithSignature to abi.encodeCall. https://github.com/OpenZeppelin/openzeppelin-contracts/pull/4293