Transaction builder

[mpizenberg]

This PR is a WIP, but ready for a first pass of reviews.

Here is a screenshot to have a quick overview of the API without having to look at the changes or using elm-doc-preview. (Might not be up-to-date with the latest state of this PR)

I’ve started by drafting all the types and writing the documentation, so that reading the documentation should tell you exactly how the building would be done in a majority of cases. I’ve also added at the end of the files some non-exposed examples in code, not only in documentation, so that the compiler would check that it makes sense.

The base idea is that there is a Tx status type, that should contain all necessary information while building the transaction. The status type parameter is a phantom type to record progress in the building and impose some very light constraints. I could have gone more type-heavy with status to have even more precise constraints, but I don’t think it’s worth it before having first some feedback.

Basically, the construction of the Tx goes as follows:

initTx
  --> Tx WIP
  |> addSomeTransfers
  |> addSomeMintsBurns
  |> addSomeScriptStuff
  |> handleChange someChangeStrat
  --> Tx AlmostReady
  |> addSomeMetadata
  |> addSomeTimeConstraints
  |> addFeeStrategy
  |> finalizeTx withSomeConfig
  --> Result String Transaction (or Task of some sort, ...)

So basically, we have the following steps:

• Initialization
• Add some intents
• Handle change
• Add some optional stuff
• Finalize the Tx

Nothing is implemented, there are basically Debug.todo everywhere, but it typechecks. Also, the finalization step I haven’t thought too much about yet. Wondering if that can stay a pure function, or eventually return a Task of some sort, that will then be sent for signature.

[mpizenberg]

To view the generated documentation, you can install elm-doc-preview and run it from the project home.

[mpizenberg]

An alternative way of building might be the following:

type alias Tx =
  { intents : List TxIntent
  , other : List TxOther
  }

finalizeTx : List Tx -> ChangeStrategy -> OtherLocalContextStuff -> Result String Transaction

type alias ChangeStrategy = List ( Output, Value ) -> ChangeReallocation

Where TxIntent would basically be the transfer|mint|plutus|... stuff, and TxOther would be the fee|time|metadata|... stuff.

Then I think it would be easier to build a Tx in small parts and join them together. Something like this?

nftMintTx =
  { intents = [ mintNft, transferToMe ]
  , other = [ nftMetadata ]
  }

swapTx =
  { intents = [ transferFromMe, sendToScript ]
  , other = [ swapMetadata ]
  }

finalizeTx [ nftMintTx, swapTx ] myChangeStrat otherContextStuff

No idea if that would be any better, or even more verbose and annoying ...

[mpizenberg]

One key design decision in this PR is the handleChange function, that is required and only called once. I did this because I thought it would enable better solutions for the transaction building problem/algorithm. Indeed, when handling intents change one-at-a-time, we do not have the full picture yet.

For example, we might add a transfer, and say to send the change back to us, but while doing something else (e.g. a swap), the change in that utxo might have been useful to use instead of sending it back and having to look for another utxo for input for the swap.

That being said, it made the last example (spending half of a utxo in a script) more challenging because we want to send the rest of that utxo back to the same script address with the same metadata. In practice, I knew the amount that had to be sent, so I could easily build the change to send back 1 ada to the contract. But what if I wanted to have it automatically figured out? I’d have to look through the whole list of consumed outputs to find the one of the script, get that amount and use it. It’s doable, just adds much more verbosity, meaning potentially bugs?

I’m wondering if a better way, would be to add "partial change" handling in the spending intents, especially the one from plutus script. Something like this:

spendFromPlutusScript :
    PlutusScriptSource
    -> ScriptUtxoSelection
    -> Value
    -> ChangeStrat -- new
    -> Tx WIP
    -> Tx WIP

Or maybe, I suspect this will only be useful if you manually select the spent utxos in the first place, but I don’t know. In that case, it could be only an addition to the manual selection type:

type ScriptUtxoSelection
    = AutoScriptUtxoSelection ({ ref : OutputReference, utxo : Output } -> Maybe { redeemer : Data })
    | ManualScriptUtxoSelection
        { selection :
            List
                { ref : OutputReference
                , utxo : Output
                , redeemer : Data
                }
        , partialChange : ChangeStrat -- new
        }

No idea if that’s better in the grand scheme of composing transactions.

[mpizenberg]

Another remark. I want to be able to accommodate a design pattern for composable transactions that rely on the position/ordering of inputs, with markers provided by the redeemer. A more complete explanation of what I mean is given by comments https://github.com/cardano-foundation/CIPs/pull/758#issuecomment-1932682215 by @colll78 and https://github.com/cardano-foundation/CIPs/pull/758#issuecomment-2250267362 by @fallen-icarus. So maybe I need to look a bit more into these contracts. Adding markers for spent script inputs that can be used in redeemers data seems like a relevant use case.

[mpizenberg]

Also another reference to keep in mind: mlab’s new purescript transaction builder https://github.com/mlabs-haskell/purescript-cardano-transaction-builder

[klntsky]

@mpizenberg consider https://github.com/klntsky/cardano-purescript altogether. the builder is useless by itself, as it is just a small dsl+interpreter

[mpizenberg]

thanks @klntsky . From what I read, the main missing part from the builder dsl is the balancing part which lives in https://github.com/Plutonomicon/cardano-transaction-lib right? I don’t have time to look at all the links in your awesome list ^^

[klntsky]

@mpizenberg yes

[mpizenberg]

@klntsky I didn’t find a way with purescript-cardano-transaction-builder to build transactions following indexing patterns, like the one-to-one input/output indexer in the redeemer pattern described in anastasia labs readme here: https://github.com/Anastasia-Labs/aiken-design-patterns?tab=readme-ov-file#singular-utxo-indexer

Do you have an idea of how that would be possible?

[mpizenberg]

Another relevant source used by Anastasia Labs https://github.com/j-mueller/sc-tools/blob/main/src/base/lib/Convex/BuildTx.hs

[klntsky]

I didn’t find a way with purescript-cardano-transaction-builder to build transactions following indexing patterns

If control over indices is needed, best to build the transaction in plain PS

[colll78]

It's not just control over the indexes that is needed, it is the ability to construct and use redeemers that depend on the indices of inputs (which can change at balancing). The redeemer should accurately reflect their correct indices as they appear in the final tx (after balancing).

[mpizenberg]

So I’ve discussed a bit with Matthias and it seems like a function-heavy API is a bit too intense, so probably a DSL approach is easier to grasp. The change control also seems mentally taxing. I liked that it enabled making sure the user is building a balanced transaction though. So I’ll probably move to a balancing approach like most others, but still not 100% sold on this. Anyway.

Taking some feedback into account, it could make sense to move to something very similar to purescript DSL approach, but with two key differences.

(1) Enabling intents based on amounts and addresses, which avoids having to select exact output references or create exact outputs for things that are fungible (so mainly assets at regular addresses or native script addresses).

(2) Enabling redeemer construction with a function that depends on the selected inputs and created outputs. Finalizing the Tx will require a multi-pass solver anyway, so I see no harm in helping dynamic redeemer constructions, as long as we protect from infinite loops in the solver.

Here is what it would look like (only describing the Spend intent as ~its the one relevant to redeemers~ I’m lazy and we can do the same with other purposes).

type Intent
    = Spend SpendSource
    | ...

type SpendSource
    = AutoSelectFrom Address Value
    | FromUtxo { input : OutputReference, spendWitness : SpendWitness }

type SpendWitness
    = NoSpendWitness
    | NativeWitness (ScriptWitness NativeScript)
    | PlutusWitness
        { scriptWitness : ScriptWitness PlutusScript
        , datumWitness : DatumWitness
        , redeemerDatum : RedeemerDatum
        }

type RedeemerDatum
    = FixedRedeemer Data
    -- This should enable easier indexing pattern
    | RedeemerFunction (RedeemerContext -> Data)

type alias RedeemerContext =
    { referenceInputs : List OutputReference
    , spentInputs : List OutputReference
    , createdOutputs : List Output
    }

[keyan-m]

Here is what it would look like (only describing the Spend intent as ~its the one relevant to redeemers~ I’m lazy and we can do the same with other purposes).

In this new API, including required signers under ScriptWitness might be a better interface than addRequiredSigners.

[mpizenberg]

Ok, I’ve refined it a bit more. Still very close to the purescript DSL but slightly different.

type TxIntent
    = SendToAutoCreate Address Value
    | SendToOutput (InputsOutputs -> Output)
      -- Spending assets from somewhere
    | SpendFromAutoSelect Address Value
    | SpendFromUtxo
        { input : OutputReference
        , spendWitness : SpendWitness
        }
      -- Minting / burning assets
    | MintBurn
        { policyId : Bytes CredentialHash
        , assets : BytesMap AssetName Integer
        , credentialWitness : CredentialWitness
        }
      -- Issuing certificates
    | IssueCertificate Todo
      -- Withdrawing rewards
    | WithdrawRewards Todo

type alias InputsOutputs =
    { referenceInputs : List OutputReference
    , spentInputs : List OutputReference
    , createdOutputs : List Output
    }

type CredentialWitness
    = NativeScriptCredential (ScriptWitness NativeScript)
    | PlutusScriptCredential
        { scriptWitness : ScriptWitness PlutusScript
        , redeemerData : InputsOutputs -> Data
        }

type SpendWitness
    = NoSpendWitness
    | NativeWitness (ScriptWitness NativeScript)
    | PlutusWitness
        { scriptWitness : ScriptWitness PlutusScript
        , datumWitness : Maybe DatumWitness
        , redeemerData : InputsOutputs -> Data
        }

type ScriptWitness a
    = ScriptValue a
    | ScriptReference OutputReference

type DatumWitness
    = DatumValue Data
    | DatumReference OutputReference

[mpizenberg]

In this new API, including required signers under ScriptWitness might be a better interface than addRequiredSigners.

@keyan-m something like this:

type ScriptWitness a
    = ScriptValue
        { script : a
        , requiredSigners : List (Bytes CredentialHash)
        }
    | ScriptReference
        { ref : OutputReference
        , requiredSigners : List (Bytes CredentialHash)
        }

[mpizenberg]

Or maybe better, I add it the plutus variants of the credential and spend witnesses to avoid the incorrect state of having required signers for a native script.

type CredentialWitness
    = NativeScriptCredential (ScriptWitness NativeScript)
    | PlutusScriptCredential
        { scriptWitness : ScriptWitness PlutusScript
        , redeemerData : InputsOutputs -> Data
        , requiredSigners : List (Bytes CredentialHash) -- new
        }

type SpendWitness
    = NoSpendWitness
    | NativeWitness (ScriptWitness NativeScript)
    | PlutusWitness
        { scriptWitness : ScriptWitness PlutusScript
        , datumWitness : Maybe DatumWitness
        , redeemerData : InputsOutputs -> Data
        , requiredSigners : List (Bytes CredentialHash) -- new
        }

[keyan-m]

What I originally had in mind was so that the signers could be shared between Native and Plutus scripts, but I hadn't really thought through that required signers of a Native script could be deduced from it directly (I understand these don't populate the required_signers field in the transaction body, but they are still required).

How about keeping only the ScriptWitness? Something like this:

type ScriptWitness
    = NativeWitness (Script NativeScript)
    | PlutusWitness
        { script : Script PlutusScript
        , datumWitness : Maybe DatumWitness
        , redeemerData : InputsOutputs -> Data
        , requiredSigners : List (Bytes CredentialHash)
        }

type Script a
    = ScriptValue a
    | ScriptReference OutputReference

This way it can be shared for all script validations (i.e. remove CredentialWitness), and SpendWitness can look like this:

type SpendWitness
  = SpendFromWallet (Bytes CredentialHash)
  | SpendFromScript ScriptWitness

Or perhaps just replacing SpendWitness with Maybe ScriptWitness? What was your reasoning behind NoSpendWitness?

[mpizenberg]

@keyan-m The NoSpendWitness is for when spending from a simple wallet output, where no witness is necessary in the transaction (the wallet signature is implied but that’s derived automatically by the node). I had copied the separation of Credential and Script witness from the purescript DSL because of the datum required for spend purposes, but I like your approach, its less repetitive. But maybe going even one step further and embedding SpendWitness info directly at the top level TxIntent gets rid of most issues.

And for CredentialWitness you suggest to merge it with ScriptWitness, and just check at runtime that people set datumWitness = Nothing everytime the script witness is provided for mint / withdrawals / certificates ?

So in the end, something like this?

EDIT: I also merged script and datum witness into WitnessSource a

type TxIntent
    = SendToAutoCreate Address Value
    | SendToOutput (InputsOutputs -> Output)
      -- Spending assets from somewhere
    | SpendFromWalletAutoSelect Address Value
    | SpendFromWallet OutputReference -- NEW
    | SpendFromScript OutputReference ScriptWitness
      -- Minting / burning assets
    | MintBurn
        { policyId : Bytes CredentialHash
        , assets : BytesMap AssetName Integer
        , credentialWitness : ScriptWitness -- CHANGED
        }
      -- Issuing certificates
    | IssueCertificate Todo
      -- Withdrawing rewards
    | WithdrawRewards
        { stakeCredential : StakeCredential
        , amount : Natural
        , credentialWitness : Maybe ScriptWitness  -- CHANGED
        }

type ScriptWitness
    = NativeWitness (WitnessSource NativeScript)
    | PlutusWitness
        { script : WitnessSource PlutusScript
        , datumWitness : Maybe (WitnessSource Data) -- NEEDS purpose check
        , redeemerData : InputsOutputs -> Data
        , requiredSigners : List (Bytes CredentialHash)
        }

type WitnessSource a
    = WitnessValue a
    | WitnessReference OutputReference

[keyan-m]

the wallet signature is implied but that’s derived automatically by the node

Ah I see, I was thinking perhaps we could detect signature-related node failures before submission.

With regards to the datum, I actually overlooked it :sweat_smile: But in your new design, perhaps we can extract it out of ScriptWitness, and put it under SpendFromScript?

-- ...
  | SpendFromScript
      { outputReference : OutputReference
      , datumWitness : Maybe (WitnessSource Data)
      , scriptWitness : ScriptWitness
      }
-- ...

type ScriptWitness
    = NativeWitness (WitnessSource NativeScript)
    | PlutusWitness
        { script : WitnessSource PlutusScript
        , redeemerData : InputsOutputs -> Data
        , requiredSigners : List (Bytes CredentialHash)
        }

[mpizenberg]

Ah I see, I was thinking perhaps we could detect signature-related node failures before submission.

I mean we can, but not via a potential requiredSigners because this semantically means that ALL signatures are required. But the native script allows for also ANY semantics and threshold semantics. That being said, when the native script is embedded by value, we could totally write a small native script interpreter right in elm-cardano. It’s relatively easy to validate I think since this is the definition:

type NativeScript
    = ScriptPubkey (Bytes NativeScriptPubkeyHash) -- should probably be (Bytes CredentialHash)?
    | ScriptAll (List NativeScript)
    | ScriptAny (List NativeScript)
    | ScriptNofK Int (List NativeScript)
    | InvalidBefore Natural
    | InvalidHereafter Natural

extract datum in SpendFromScript

Ok, makes sense I think.

[klntsky]

(2) Enabling redeemer construction with a function that depends on the selected inputs and created outputs. Finalizing the Tx will require a multi-pass solver anyway, so I see no harm in helping dynamic redeemer constructions, as long as we protect from infinite loops in the solver.

This is interesting, I would like to see it implemented. But I think you can generalize it even more: let the user supply a function that turns any transaction into a transaction that satisfies some invariant the user cares about, and then plug it into the balancer loop.

[mpizenberg]

I think you can generalize it even more: let the user supply a function that turns any transaction into a transaction that satisfies some invariant the user cares about, and then plug it into the balancer loop.

Possible indeed. I guess we will try restricted power first and see how that goes while building as many examples as possible. Maybe it turns out annoying if some setup is repeated over for each redeemer, in that case it might turn out better to do something like finalize : (InputsOutputs -> List TxIntent) ...

[mpizenberg]

Ok, I’ve made the changes towards the DSL approach. I’ve also updated the Tx building doc with the examples to show what it would look like. To view the new documentation, you can simply run elm-doc-preview from the repo root, and open the Cardano module page with the link on the right of the page docs.

Let me know what you think. It’s not necessarily the final design, I still have some discussions to have. But if it’s good enough for a first try, I’ll proceed with some implementation for finalize.

[mpizenberg]

Ok, I feel like this PR has gone far enough to be merged, even if incomplete. The current state of Tx building is the following.

The building API has converged to a DSL approach, with two specificities. (1) There is a Spend <| From address value variant and a SendTo address value variant that enable loose specification of inputs and outputs, that can be leveraged by the coin selection algorithm. (2) Redeemers are specified via a function of the shape InputsOutputs -> ... Redeemer ... enabling redeemer values to be constructed by inspecting the list of inputs and outputs. This is especially useful for patterns like the UTxO indexing described in Anastasia Lab’s aiken patterns repo.

This PR also provides a first implementation of the Tx builder finalization step. That step processes all intents, performs coin selection and build a balanced Transaction. Current limitations include the fact that Tx fees and script costs are not estimated yet, so the balancing is done without. But it would not fundamentally change how the current code works so this doesn’t contradict this proof of concept. The main difference in the future follow up PR, is that it will need to define side effects and ports communication to finalize the Tx with correct fees and script costs.

Another limitation is that this PR does not perform blake2b hashing yet for the different fields that require them.

Finally, this PR includes some example code in the bottom of the Cardano module, as well as in a dedicated examples/txbuild/ folder. It temporarily exposes some values, for testing purposes that will eventually be removed or moved elsewhere.

That’s all :)

Enjoy this screenshot from the txbuild example.