Scale: Improve the scalability of Plutus Minting Policy

[thaddeusdiamond]

Last one before I deploy :-) Looking to improve the scalability of my minting policy. Currently, it can mint 9 NFTs at a time, which might be fine. But if someone is trying to mint 150 "ballots" (I have a couple of whales), they will have to sign 17 transactions to record their votes. I would love to scale the minting policy below to 30 NFTs at once (beyond that we run into transaction limits anyway).

minting voting_ballot

const BALLOT_BOX_PUBKEY: ValidatorHash = ValidatorHash::new(#${pubKeyHash})
const BALLOT_NAME_PREFIX: ByteArray = #${ballotPrefix}
const POLLS_CLOSE: Time = Time::new(${pollsClose})
const REFERENCE_POLICY_HASH: MintingPolicyHash = MintingPolicyHash::new(#${referencePolicyId})
const SINGLE_NFT: Int = 1

enum Redeemer {
  Mint
  Burn
}

func assets_locked_in_script(tx: Tx, minted_assets: Value) -> Bool {
  //print(tx.value_sent_to(BALLOT_BOX_PUBKEY).serialize().show());
  //print(minted_assets.serialize().show());
  ballots_sent: Value = tx.value_locked_by(BALLOT_BOX_PUBKEY);
  assets_locked: Bool = ballots_sent.contains(minted_assets);
  if (assets_locked) {
    true
  } else {
    print("Minted ballots (" + minted_assets.serialize().show() + ") were not correctly locked in the script: " + ballots_sent.serialize().show());
    false
  }
}

func tx_outputs_contain(voting_asset: AssetClass, outputs: []TxOutput) -> Bool {
  outputs.any((tx_out: TxOutput) -> Bool {
    //print("Searching...");
    //print(voting_asset.serialize().show());
    //print(tx_out.value.serialize().show());
    tx_out.value.contains(Value::new(voting_asset, SINGLE_NFT))
  })
}

func assets_were_spent(minted_assets: Value, policy: MintingPolicyHash, outputs: []TxOutput) -> Bool {
  tx_sends_to_self: Bool = minted_assets.get_policy(policy).all((asset_id: ByteArray, amount: Int) -> Bool {
    original_asset_name: ByteArray = asset_id.slice(BALLOT_NAME_PREFIX.length, asset_id.length);
    voting_asset: AssetClass = AssetClass::new(REFERENCE_POLICY_HASH, original_asset_name);
    tx_outputs_contain(voting_asset, outputs) && amount == SINGLE_NFT
  });
  if (tx_sends_to_self) {
    true
  } else {
    print("The NFTs with voting power (" + REFERENCE_POLICY_HASH.serialize().show() + ") for the ballots were never sent-to-self");
    false
  }
}

func polls_are_still_open(time_range: TimeRange) -> Bool {
  tx_during_polls_open: Bool = time_range.is_before(POLLS_CLOSE);
  if (tx_during_polls_open) {
    true
  } else {
    print("Invalid time range: " + time_range.serialize().show() + " (polls close at " + POLLS_CLOSE.serialize().show() + ")");
    false
  }
}

func main(redeemer: Redeemer, ctx: ScriptContext) -> Bool {
  tx: Tx = ctx.tx;
  minted_policy: MintingPolicyHash = ctx.get_current_minting_policy_hash();
  redeemer.switch {
    Mint => {
      polls_are_still_open(tx.time_range)
        && assets_were_spent(tx.minted, minted_policy, tx.outputs)
        && assets_locked_in_script(tx, tx.minted)
    },
    Burn => {
      tx.minted.get_policy(minted_policy).all((asset_id: ByteArray, amount: Int) -> Bool {
        if (amount > 0) {
          print(asset_id.show() + " asset ID was minted not burned (quantity " + amount.show() + ")");
          false
        } else {
          true
        }
      })
    }
  }

The mem budget is what is exhausted first (the prints are never triggered). Suggestions I have heard before:

• Compile using Plutonomy to do more efficient type constraints
• Use new Plutus cost models

I have not looked into whether Helios is doing either of these. Let me know your advice. Full error trace below

{
 \"ScriptFailures\": {
 \"mint:0\": [
 {
 \"validatorFailed\": {
 \"error\": \"An error has occurred: User error:\
The machine terminated part way through evaluation due to overspending the budget.\
The budget when the machine terminated was:\
({ cpu: 3179282965\
| mem: -1752\
})\
Negative numbers indicate the overspent budget; note that this only indicatessthe budget that was needed for the next step, not to run the program to completion.\",
 \"traces\": []
 }
 }
 ]
 }
}

[christianschmitz]

My first thought is that this looks like a budget miscalculation error, although then the remaining cpu budget should be much closer to 0.

If you compile with optimize=true then the resulting plutus-core should be quite optimal (and nothing will be printed because that's optimized out). To compile with optimize=true: let uplcProgram = Program.new(<src>).compile(true)

A minor optimization that I can think of, that isn't automatically performed by the compiler when optimize=true, is returning the condition of if-else expressions directly, instead of eg. if (cond) {true} else {false}.

The most expensive part of the script is calling minted_assets.get_policy(policy).all(...) (inside assets_were_spent()), and inside that loop calling outputs.any() (inside tx_outputs_contain()), and then inside that loop calling tx_out.value.contains(...). This is essentially a quadruple nested loop (a Value is a nested map) which gets more and more expensive as the outputs contain more entries. Many of the iterations are redundant, I'll try to think of way to do this more optimally

[thaddeusdiamond]

Yes unfortunately the inner loop is O(n*m) where n is the number of tx_outputs and m is the number of minted_assets. Not sure how you can get around this (and from a storage perspective there's no optimization as it is O(1) in storage just accumulating the ongoing boolean. I will try the optimize flags you suggested.

[thaddeusdiamond]

Okay went and did optimize, Blockfrost is returning nothing on the Tx Evaluate so the transaction cannot complete:

curl -X POST -H 'project_id: <BLOCKFROST_PROJ>' https://cardano-mainnet.blockfrost.io/api/v0/utils/txs/evaluate --data-binary @./data -H 'Content-Type: application/cbor'

CBOR:

84a80082825820045d1b1e6b3d997b6e190880ded746305a9fc0fbc20da522c688bcdfddc3606701825820c2dd94a669ff2052c8f39b98711ba51d69557a0e6703ddceab6dd03a3d681230020184a300581d717c662e710adf2dd0343c53259977b8e573ab784c72bc4d98ea9d750a01821a0026eee0a1581cdaf3bb326b7820b351ddbb4ee91096a921d81eee46fa11f060055809aa55566f7465202331202d2050657473525573303034300155566f7465202331202d2050657473525573303732310155566f7465202331202d2050657473525573303734370155566f7465202331202d2050657473525573303831340155566f7465202331202d2050657473525573303831350155566f7465202331202d2050657473525573303831390155566f7465202331202d2050657473525573303832300155566f7465202331202d2050657473525573303834320155566f7465202331202d2050657473525573313033330155566f7465202331202d20506574735255733130333601028201d818587ba245766f7465725f5840616464723171796b6a683978726a3536366a6d6d35656d3065707a6d73666d646676707738366e6e6333673866377a736e38337536753361396c6d337a71787758276736736e326d3933636d306c657174616d3661707a6a65343578637739647135733675386c3333ff44766f74654142825839012d2b94c39535a96f74cedf908b704eda9605c7d4e788a0e9f0a133c79ae47a5fee22019c8d426ad9638dbff902fbbd7422966b4361c56829821a00198458a1581c6eefdae9f6a7214b53c8d99769955f11dd5d9b12f731eeb0ff6b12d9aa4b5065747352557330303430014b5065747352557330373231014b5065747352557330373437014b5065747352557330383134014b5065747352557330383135014b5065747352557330383139014b5065747352557330383230014b5065747352557330383432014b5065747352557331303333014b506574735255733130333601825839012d2b94c39535a96f74cedf908b704eda9605c7d4e788a0e9f0a133c79ae47a5fee22019c8d426ad9638dbff902fbbd7422966b4361c56829821a00cab05cb6581c04758324d8e5fef55a2e1780f986317a4581952261ade74ce8b634a1a1581a53747566662068617070656e6564206f6e20746865206d656e7501581c051c7a76ad8ceaccd3d077ab616a7073e1620adfd6fb3e1839723f65a15143617264616e6f496e436f6c6f7245363001581c1e12e6a56eaa7ba5c6da86f8bc33b8eacce03b80d4d16e685255694ea255596f756e67416461546865546565323032323332340155596f756e674164615468655465653230323233373201581c24a32f531db96d0b2daf673d31e6203c94eebdc83781d7f9d5dc6ae5a14574616e677a01581c2521b39414dacf3acef85cd1cb316ef092d94d6e53b3a9c98ea6a288a14f416c7068614163636573733130333201581c33568ad11f93b3e79ae8dee5ad928ded72adcea719e92108caf1521ba74d57696c6454616e677a20333833014d57696c6454616e677a20353836014d57696c6454616e677a20373633014d57696c6454616e677a20393532014e57696c6454616e677a2031363031014e57696c6454616e677a2032373135014e57696c6454616e677a203431323301581c420398d5cb09b8663843718991aca13b17113766f4209c88ee577a37a14654434b54323401581c44b84d95018f9898567a1017ca4d99f3247a18824136e45844ab6cb6a5581d55576561706f6e73476c61646961746f7247656e65736973303333303901581d55576561706f6e73476c61646961746f7247656e65736973303931383101581d55576561706f6e73476c61646961746f7247656e65736973313730303901581d55576561706f6e73476c61646961746f7247656e65736973323332323301581d55576561706f6e73476c61646961746f7247656e65736973323435313801581c624114e85e91604cc8aced7c7685842d7fd6ba63b733beee9ecc09b1a24954726f676730383833014954726f67673630303101581c6d6b6f368b2e9a49ee00ff919d76ffed11b838a1988f4f888f420a22a1581e5374616720416c6c69616e636520506f7274616c2050617373202330353401581c6eefdae9f6a7214b53c8d99769955f11dd5d9b12f731eeb0ff6b12d9b86a4b5065747352557331303730014b5065747352557331333036014b5065747352557331333039014b5065747352557331373335014b5065747352557331373630014b5065747352557331373637014b5065747352557331373738014b5065747352557331383131014b5065747352557331383232014b5065747352557331383437014b5065747352557331383833014b5065747352557331393030014b5065747352557331393738014b5065747352557332303031014b5065747352557332303233014b5065747352557332313439014b5065747352557332323033014b5065747352557332323932014b5065747352557332333635014b5065747352557332333930014b5065747352557332343136014b5065747352557332343638014b5065747352557332343830014b5065747352557332353136014b5065747352557332353335014b5065747352557332353634014b5065747352557332363135014b5065747352557332363136014b5065747352557332363338014b5065747352557332363539014b5065747352557332373039014b5065747352557332373138014b5065747352557332373231014b5065747352557332373537014b5065747352557332373631014b5065747352557332373936014b5065747352557332383232014b5065747352557332383432014b5065747352557332383438014b5065747352557332393433014b5065747352557333323334014b5065747352557333323536014b5065747352557333323537014b5065747352557333323538014b5065747352557333323637014b5065747352557333323730014b5065747352557333323838014b5065747352557333333338014b5065747352557333333434014b5065747352557333333538014b5065747352557333333735014b5065747352557333333736014b5065747352557333333832014b5065747352557333333939014b5065747352557333343033014b5065747352557333353331014b5065747352557333383839014b5065747352557333393333014b5065747352557334303230014b5065747352557334303639014b5065747352557334313033014b5065747352557334313034014b5065747352557334313439014b5065747352557334313539014b5065747352557334313830014b5065747352557334323432014b5065747352557334323437014b5065747352557334323731014b5065747352557334323931014b5065747352557334333136014b5065747352557334333436014b5065747352557334333538014b5065747352557334333932014b5065747352557334343135014b5065747352557334343138014b5065747352557334343835014b5065747352557334343938014b5065747352557334353132014b5065747352557334353439014b5065747352557334363334014b5065747352557334363739014b5065747352557334363834014b5065747352557334373438014b5065747352557334373633014b5065747352557334373830014b5065747352557334383533014b5065747352557334383537014b5065747352557334383838014b5065747352557334393139014b5065747352557334393233014b5065747352557334393536014b5065747352557335303330014b5065747352557335303333014b5065747352557335303437014b5065747352557335313130014b5065747352557335313334014b5065747352557335313935014b5065747352557335323031014b5065747352557335323234014b5065747352557335323236014b5065747352557335323735014b5065747352557335333239014b5065747352557335333637014b5065747352557335333733014b5065747352557335343733014b506574735255733535303101581c6f87aa2ea552ecd24732d62831916ece54c52348417398085358cbeda15453746f6e65416765486f6f6c6967616e3032303501581c8fe3507c67a24530f686fbed62abc56ad55625c7d4d8619a04f5ed0ba14f596f6c6f536f6c6f734f473135343601581c9f982855bd908ae21a423d66a68a85002a9e44da9919c60311472dd2a64753464d43333439014753464d43343731014753464d43363634014853464d4331303138014853464d4331313635014853464d433131393101581cb56700386db953b7dfa95613e09c0ab89ea9a5bb447b0dc1f8777777a14b4d6f6e636861696e35333401581cb5fb333ad44e4853484cef178f543c35564490534aafcdf919fa4687a1443138333801581cbe2548e33981e2f6455d651f6d9094405a7b92b0c461aeea8ecb14aaa14c476f6464657373202331333501581cc0e6c802993aadbe53fccf26811d21965854109dc8d8ece113cca940a44d436c6179427566667932393330014d436c6179427566667933383238014d436c6179427566667934363836014d436c617942756666793438353201581cc281975562f394761771f13f599881517fa8455946e7e30454de22daa14e474f41545472696265303532353101581cc53df5abd223e263ee82538ed6682ae5669b033068cbae6eb1b21059a1443037373001581cee23dd7fc5512d5de1ce33c06b06f739d44d7bb5c206bd010ca87b4fa5474247433036333501474247433036343901474247433036353501474247433036373001474247433036383801581cf0ff48bbb7bbe9d59a40f1ce90e9e9d0ff5002ec48f232b49ca0fb9aa74863616d6f6c746f65014877696c6474616e67014977696c6474616e6773014977696c6474616e677a014c77696c6474616e677a2e696f014e77696c6474616e677a7072697a65014f77696c6474616e677a7072697a657301825839012d2b94c39535a96f74cedf908b704eda9605c7d4e788a0e9f0a133c79ae47a5fee22019c8d426ad9638dbff902fbbd7422966b4361c568291a00779563021a0007cdf1031a0462f4590758204f58c89097ca8ad1f7df97644de124a81edbc98bad3bdb6424812c487d41379409a1581cdaf3bb326b7820b351ddbb4ee91096a921d81eee46fa11f060055809aa55566f7465202331202d2050657473525573303034300155566f7465202331202d2050657473525573303732310155566f7465202331202d2050657473525573303734370155566f7465202331202d2050657473525573303831340155566f7465202331202d2050657473525573303831350155566f7465202331202d2050657473525573303831390155566f7465202331202d2050657473525573303832300155566f7465202331202d2050657473525573303834320155566f7465202331202d2050657473525573313033330155566f7465202331202d205065747352557331303336010b58205b09f3819069340313c5e997b112443db61871cce52e43c2abf133b9f60c59360e81581c2d2b94c39535a96f74cedf908b704eda9605c7d4e788a0e9f0a133c7a20581840100d87980820000068159058859058501000032323232323232323232323232323232322233335734646466666ae68cdc39aab9d37540089000111980b91980c11999ab9a3232323333573466e1c005200424a046666ae68cdc38012400049448cccd5cd199ab9a3370e6aae74dd51809802a400094128919b88375a6ae84d55cf1baa005482034a66f3f9808cdc49bad357426aae78dd5002a4101a53379fcc09324c931aab9d37540026020002601c6ae84d5d11aba235744602a00c494492824c46666ae68cc048cc044c05001801488cc06c8cc040dd6180b0049198069809000991ba633574066ec0c050004dd319aba03376060260029810101004bd6f7b63025eb7bdb180cdd2a400066ae8130011e581c6eefdae9f6a7214b53c8d99769955f11dd5d9b12f731eeb0ff6b12d900335740646ea4ccdc62402866e04c8cccd5cd19b88001480008cdc019b80371a0069001001100124c6e34dd7002a40280026eb80112f5c0466ebc00930101010024a249412623333573466010646666002002646660020026eb0c05001c8c8cccd5cd19b8735573a6ea8005200223375e98011e581c7c662e710adf2dd0343c53259977b8e573ab784c72bc4d98ea9d750a0030133333573466e1cd55ce9baa00248008800c992624a093180898088009111999ab9a35746004497ae023333573460046ae8400c8cd5d01aba100433300500535744008006466600a00a6ae8801000d2649888cc88c8dd31998008009980880180125eb7bdb180888cccd5cd1aba300220022332232333357346ae8c004800c8cd5d019bb0004374c004006931991191998008009980c80180125eb7bdb180888cccd5cd1aba3002200223322323333573466e1c00520002003233574066ec0010dd4001001a4c66e00cc074020008cc07401c008d5d08019998020021aba2003002498cc050020008cc05001c008d5d08019998020021aba2003002498dd58011bab30100014c101a0002222333357346ae8c00c80088cccc014014d5d1002001998018011aba1004498c04c014928925049888cc040cc03cc04801000c88cccd5cd180819b89375a0029000125024a29324c0086014601200460120024931324c446644646600200266010006004446666ae68d5d1800925123233335734601e664464660020026601c006004446666ae68d5d1800925123233335734602a602a66e20cc040018004cc0400140049281198028029aba2004498d5d080124c6601200c0026601200a00249408cc014014d5d100224c6ae840092637560046eac00488c8cc00400400c88cccd5cd1aba300124bd6f7b63011999ab9a3375e6aae74d5d080100211bab35573c6ae8400c8cc010010d5d1001a4c9311191998008008018011111999ab9a357460044900011999ab9a3375e6aae74d5d080180111bad35573c6ae840108ccc014014d5d1002001a4c93111919191998008009998010010018020019111999ab9a357460024006466ae80d5d08011998020020019aba2002498888cccd5cd1aba300124bd701191999ab9a3300900623375e002004466600c00c00a6ae880108cd5d00011998030030029aba2004498d55ce9aba1002498c8cc00400400c88cccd5cd1aba300124bd70119aba035573a6ae84008cc00c00cd5d100124c44646660020020060044446666ae68d5d1801125023333573460046ae8400c92891998028029aba200400349926235742601400246ae84c0280048ccd5cd000a504a24464660020026eac00c88cccd5cd1aba30012623333573466ebcd55ce9aba1002004235573c6ae8400c8cc010010d5d1001a4c9311191998008009bab00323300335573a0026aae78004888cccd5cd1aba300224a246666ae68c008d5d080191998028029aba200400324a09324c46ae84c0100048d5d0980100091aba235744600400246ae88c0080048d5d1180100091aab9e3754002446666ae68c009262300249892824df5a11902d1a278386461663362623332366237383230623335316464626234656539313039366139323164383165656534366661313166303630303535383039aa75566f7465202331202d205065747352557330303430a66b6465736372697074696f6e827824436f6d6d656d6f726174697665204e465420666f72206f6e2d636861696e20766f746520783d284d6574616461746120666f7220636f6e76656e69656e63652c2066696e616c20766f746573207265636f72646564207573696e6720646174756d732965696d61676567697066733a2f2f696d656469615479706569696d6167652f706e67646e616d6575566f7465202331202d2050657473525573303034306770726f6a656374781b53616d706c652042616c6c6f74202d204e46542050726f6a65637464766f7465614275566f7465202331202d205065747352557330373231a66b6465736372697074696f6e827824436f6d6d656d6f726174697665204e465420666f72206f6e2d636861696e20766f746520783d284d6574616461746120666f7220636f6e76656e69656e63652c2066696e616c20766f746573207265636f72646564207573696e6720646174756d732965696d61676567697066733a2f2f696d656469615479706569696d6167652f706e67646e616d6575566f7465202331202d2050657473525573303732316770726f6a656374781b53616d706c652042616c6c6f74202d204e46542050726f6a65637464766f7465614275566f7465202331202d205065747352557330373437a66b6465736372697074696f6e827824436f6d6d656d6f726174697665204e465420666f72206f6e2d636861696e20766f746520783d284d6574616461746120666f7220636f6e76656e69656e63652c2066696e616c20766f746573207265636f72646564207573696e6720646174756d732965696d61676567697066733a2f2f696d656469615479706569696d6167652f706e67646e616d6575566f7465202331202d2050657473525573303734376770726f6a656374781b53616d706c652042616c6c6f74202d204e46542050726f6a65637464766f7465614275566f7465202331202d205065747352557330383134a66b6465736372697074696f6e827824436f6d6d656d6f726174697665204e465420666f72206f6e2d636861696e20766f746520783d284d6574616461746120666f7220636f6e76656e69656e63652c2066696e616c20766f746573207265636f72646564207573696e6720646174756d732965696d61676567697066733a2f2f696d656469615479706569696d6167652f706e67646e616d6575566f7465202331202d2050657473525573303831346770726f6a656374781b53616d706c652042616c6c6f74202d204e46542050726f6a65637464766f7465614275566f7465202331202d205065747352557330383135a66b6465736372697074696f6e827824436f6d6d656d6f726174697665204e465420666f72206f6e2d636861696e20766f746520783d284d6574616461746120666f7220636f6e76656e69656e63652c2066696e616c20766f746573207265636f72646564207573696e6720646174756d732965696d61676567697066733a2f2f696d656469615479706569696d6167652f706e67646e616d6575566f7465202331202d2050657473525573303831356770726f6a656374781b53616d706c652042616c6c6f74202d204e46542050726f6a65637464766f7465614275566f7465202331202d205065747352557330383139a66b6465736372697074696f6e827824436f6d6d656d6f726174697665204e465420666f72206f6e2d636861696e20766f746520783d284d6574616461746120666f7220636f6e76656e69656e63652c2066696e616c20766f746573207265636f72646564207573696e6720646174756d732965696d61676567697066733a2f2f696d656469615479706569696d6167652f706e67646e616d6575566f7465202331202d2050657473525573303831396770726f6a656374781b53616d706c652042616c6c6f74202d204e46542050726f6a65637464766f7465614275566f7465202331202d205065747352557330383230a66b6465736372697074696f6e827824436f6d6d656d6f726174697665204e465420666f72206f6e2d636861696e20766f746520783d284d6574616461746120666f7220636f6e76656e69656e63652c2066696e616c20766f746573207265636f72646564207573696e6720646174756d732965696d61676567697066733a2f2f696d656469615479706569696d6167652f706e67646e616d6575566f7465202331202d2050657473525573303832306770726f6a656374781b53616d706c652042616c6c6f74202d204e46542050726f6a65637464766f7465614275566f7465202331202d205065747352557330383432a66b6465736372697074696f6e827824436f6d6d656d6f726174697665204e465420666f72206f6e2d636861696e20766f746520783d284d6574616461746120666f7220636f6e76656e69656e63652c2066696e616c20766f746573207265636f72646564207573696e6720646174756d732965696d61676567697066733a2f2f696d656469615479706569696d6167652f706e67646e616d6575566f7465202331202d2050657473525573303834326770726f6a656374781b53616d706c652042616c6c6f74202d204e46542050726f6a65637464766f7465614275566f7465202331202d205065747352557331303333a66b6465736372697074696f6e827824436f6d6d656d6f726174697665204e465420666f72206f6e2d636861696e20766f746520783d284d6574616461746120666f7220636f6e76656e69656e63652c2066696e616c20766f746573207265636f72646564207573696e6720646174756d732965696d61676567697066733a2f2f696d656469615479706569696d6167652f706e67646e616d6575566f7465202331202d2050657473525573313033336770726f6a656374781b53616d706c652042616c6c6f74202d204e46542050726f6a65637464766f7465614275566f7465202331202d205065747352557331303336a66b6465736372697074696f6e827824436f6d6d656d6f726174697665204e465420666f72206f6e2d636861696e20766f746520783d284d6574616461746120666f7220636f6e76656e69656e63652c2066696e616c20766f746573207265636f72646564207573696e6720646174756d732965696d61676567697066733a2f2f696d656469615479706569696d6167652f706e67646e616d6575566f7465202331202d2050657473525573313033366770726f6a656374781b53616d706c652042616c6c6f74202d204e46542050726f6a65637464766f746561426776657273696f6e63312e30

[christianschmitz]

Weird, I thought blockfrost always returns an error-message if the tx can't be submitted.

You're already confident enough to submit it to mainnet?

[thaddeusdiamond]

Not really I just have 100+ of the same policy on mainnet so was testing that scale. Don't have 100 test NFTs. Let me see what happens when I do this without optimize flags...

Neither one works. This seems to be a blockfrost issue because it works fine on preprod. Will keep you posted.

[thaddeusdiamond]

Setting optimize to true only improved the memory usage by 1800 units, so very little impact (still limited to 9 NFTs). Are we using the latest Plutus cost models? Any easy way to check that in the CBOR?

[christianschmitz]

The cost model parameters are hashed and must correspond with the most recent network parameters, otherwise you wouldn't be able to submit anything (and there should be a clear error-message).

Which library are using to submit the tx, Lucid or Helios or other? Each library calculates the execution budget independently so you could check one against the other. I'm fairly confident the Helios execution budget calculation is correct.

Another question: what is the use of the assets_were_spent() check? Transactions need to be balanced so all minted nfts must be in the outputs anyway. The only use I can think of is to make sure that each output only contains one minted nft, but that can be done much more efficiently.

[thaddeusdiamond]

That is checking that the user doing the mint spent the reference assets. So if you minted “Ballot #1 - WildTangz 12” you also sent yourself “WildTangz 12”.  It’s creating proxy NFT ballots based on your existing ownership.

[christianschmitz]

I understand, it's a token proving voting eligibility, like a voter id.

In that case I would put the minted ballot tokens and the outputs returning the voter id tokens in the same order when building the tx (so those outputs come first), and then the script can simply loop through both at the same time and compare.

You would have to do a low-level recursive loop though, using head, tail, and is_empty(), because there are no builtin methods that allow looping through two or more lists at the same time.

[thaddeusdiamond]

Can that order be guaranteed? And even if it is you will not save mem (in fact you might have to keep a mem overhead for the pointer to list entry you are currently comparing).

[christianschmitz]

Yes, order is guaranteed.

As I mentioned before the functions you are currently using essentially form a quadruple nested loop, so not very efficient.

Mem budget is related to memory required to run a builtin function (it is cumulative because it assumes memory isn't freed), not memory required to store variables. So less nesting levels in a loop -> less function calls -> less mem/cpu usage.

The weird thing though is that the cpu and mem usage you reported aren't proportional (if anything cpu should run out before mem, most builtin functions have constant mem cost), so there could also be something wrong with the way the budget is calculated. Can you confirm you are still using Lucid for building the tx?

[thaddeusdiamond]

Yes I am still using Lucid but the evaluation happens on the blockfrost endpoint since nativeUplc is set to false.

[thaddeusdiamond]

Okay here is another attempt, but it doesn't compile:

minting voting_ballot

const BALLOT_BOX_PUBKEY: ValidatorHash = ValidatorHash::new(#${pubKeyHash})
const BALLOT_NAME_PREFIX: ByteArray = #${ballotPrefix}
const POLLS_CLOSE: Time = Time::new(${pollsClose})
const REFERENCE_POLICY_HASH: MintingPolicyHash = MintingPolicyHash::new(#${referencePolicyId})
const SINGLE_NFT: Int = 1

enum Redeemer {
  Mint
  Burn
}

func assets_locked_in_script(tx: Tx, minted_assets: Value) -> Bool {
  //print(tx.value_sent_to(BALLOT_BOX_PUBKEY).serialize().show());
  //print(minted_assets.serialize().show());
  ballots_sent: Value = tx.value_locked_by(BALLOT_BOX_PUBKEY);
  assets_locked: Bool = ballots_sent.contains(minted_assets);
  if (assets_locked) {
    true
  } else {
    print("Minted ballots (" + minted_assets.serialize().show() + ") were not correctly locked in the script: " + ballots_sent.serialize().show());
    false
  }
}

/*func tx_outputs_contain(voting_asset: AssetClass, outputs: []TxOutput) -> Bool {
  outputs.any((tx_out: TxOutput) -> Bool {
    //print("Searching...");
    //print(voting_asset.serialize().show());
    //print(tx_out.value.serialize().show());
    tx_out.value.contains(Value::new(voting_asset, SINGLE_NFT))
  })
}*/

func assets_were_spent(minted: Value, policy: MintingPolicyHash, outputs: []TxOutput) -> Bool {
  minted_assets: Map[ByteArray]Int = minted.get_policy(policy);
  reference_assets_map = minted_assets.map((asset_id: ByteArray, amount: Int) -> Map {
    original_asset_name: ByteArray = asset_id.slice(BALLOT_NAME_PREFIX.length, asset_id.length);
    voting_asset: AssetClass = AssetClass::new(REFERENCE_POLICY_HASH, original_asset_name);
    (voting_asset, amount)
  });
  tx_sends_to_self: Bool = outputs.any((tx_out: TxOutput) -> Bool {
    minted_assets_map.all((asset_id: ByteArray, amount: Int) -> Bool {
      tx_out.value.contains(Value::new(voting_asset, amount))
    })
  })
  if (tx_sends_to_self) {
    true
  } else {
    print("The NFTs with voting power (" + REFERENCE_POLICY_HASH.serialize().show() + ") for the ballots were never sent-to-self");
    false
  }
}

func polls_are_still_open(time_range: TimeRange) -> Bool {
  tx_during_polls_open: Bool = time_range.is_before(POLLS_CLOSE);
  if (tx_during_polls_open) {
    true
  } else {
    print("Invalid time range: " + time_range.serialize().show() + " (polls close at " + POLLS_CLOSE.serialize().show() + ")");
    false
  }
}

func main(redeemer: Redeemer, ctx: ScriptContext) -> Bool {
  tx: Tx = ctx.tx;
  minted_policy: MintingPolicyHash = ctx.get_current_minting_policy_hash();
  redeemer.switch {
    Mint => {
      polls_are_still_open(tx.time_range)
        && assets_were_spent(tx.minted, minted_policy, tx.outputs)
        && assets_locked_in_script(tx, tx.minted)
    },
    Burn => {
      tx.minted.get_policy(minted_policy).all((asset_id: ByteArray, amount: Int) -> Bool {
        if (amount > 0) {
          print(asset_id.show() + " asset ID was minted not burned (quantity " + amount.show() + ")");
          false
        } else {
          true
        }
      })
    }
  }
}

When I use the Helios Playground compile returns no visible errors, but I see this in the developer tools:

EditorTab.js:71 Uncaught Error: unexpected undefined value
    at assertDefined (helios.js:307:9)
    at buildMapTypeExpr (helios.js:14557:34)
    at buildTypeExpr (helios.js:14524:10)
    at buildFuncLiteralExpr (helios.js:14314:20)
    at buildChainStartValueExpr (helios.js:14866:10)
    at buildChainedValueExpr (helios.js:14832:13)
    at Array.exprBuilders (helios.js:14671:11)
    at buildValueExpr (helios.js:14675:27)
    at Array.<anonymous> (helios.js:14820:11)
    at buildValueExpr (helios.js:14675:27)
react-dom.development.js:4299 Uncaught Error: unexpected undefined value
    at assertDefined (helios.js:307:9)
    at buildMapTypeExpr (helios.js:14557:34)
    at buildTypeExpr (helios.js:14524:10)
    at buildFuncLiteralExpr (helios.js:14314:20)
    at buildChainStartValueExpr (helios.js:14866:10)
    at buildChainedValueExpr (helios.js:14832:13)
    at Array.exprBuilders (helios.js:14671:11)
    at buildValueExpr (helios.js:14675:27)
    at Array.<anonymous> (helios.js:14820:11)
    at buildValueExpr (helios.js:14675:27

[thaddeusdiamond]

Okay so that looks to be an error with the playground, when I changed the line in the code to:

reference_assets_map: Map[ByteArray]Int = minted_assets.map((asset_id: ByteArray, amount: Int) -> Map[ByteArray]Int {
...

It started producing compile errors again.

[thaddeusdiamond]

@christianschmitz: Any way you can expose the map function in a map type so I can transform only once? I see __helios__map__map in the code here, but it is not exposed in the MapType class here. That might simplify the CPU for this function down from O(n*m) using properly ordered all and any:

func assets_were_spent(minted: Value, policy: MintingPolicyHash, outputs: []TxOutput) -> Bool {
  minted_assets: Map[ByteArray]Int = minted.get_policy(policy);
  // The line below this comment fails with "function not found"
  reference_assets_map: Map[ByteArray]Int = minted_assets.map((asset_id: ByteArray, amount: Int) -> Map[ByteArray]Int {
    original_asset_name: ByteArray = asset_id.slice(BALLOT_NAME_PREFIX.length, asset_id.length);
    voting_asset: AssetClass = AssetClass::new(REFERENCE_POLICY_HASH, original_asset_name);
    mkPairData(voting_asset, amount)
  });
  tx_sends_to_self: Bool = outputs.any((tx_out: TxOutput) -> Bool {
    minted_assets_map.all((asset_id: ByteArray, amount: Int) -> Bool {
      tx_out.value.contains(Value::new(voting_asset, amount))
    })
  });
  if (tx_sends_to_self) {
    true
  } else {
    print("The NFTs with voting power (" + REFERENCE_POLICY_HASH.serialize().show() + ") for the ballots were never sent-to-self");
    false
  }
}

[christianschmitz]

Okay so that looks to be an error with the playground, when I changed the line in the code to:

reference_assets_map: Map[ByteArray]Int = minted_assets.map((asset_id: ByteArray, amount: Int) -> Map[ByteArray]Int {
...

It started producing compile errors again.

The MapTypeExpr build function was throwing regular errors instead of UserError in case of bad syntax. I've fixed that now (use following url for latest version of playground: https://www.hyperion-bt.org/Helios-Playground/).

I think the reason I didn't yet expose the Map.map method is because I wasn't sure what the return type should be of the inner function (Helios doesn't have tuples). I guess I could expose a map function that only maps keys, and a map function that only maps values. I think map_keys would work in your case because you don't care about amount.

[thaddeusdiamond]

map_keys would be fine, because if you really needed access to the values you could use get or get_safe (if the internal implementation is a hash table it would run a lookup in O(1) roughly).

[christianschmitz]

I've just added Map.map_keys() and Map.map_values(), and I've removed tx.now() in favor of tx.time_range.start and tx.time_range.end.

Sadly a Map in plutus-core isn't implemented as a hash-table, but is just a simple list (so duplicate entries are possible).

Other changes:

• an if-else expression with boolean cases is now optimized to just the condition where possible
• some tx-building methods have been renamed and take other arguments

[thaddeusdiamond]

Okay nicely done! All these improvements and rewritten code have me up to 12 at a time: a 33% improvement! The only thing I could think of that might help lastly is if I can construct a single Value with multiple asset classes. My lucid Tx already constructs the payToAddress for self-send in one UTxO index.

I don't see the multi-asset constructor. Here is my Helios function now:

func assets_were_spent(minted: Value, policy: MintingPolicyHash, outputs: []TxOutput) -> Bool {
  minted_assets: Map[ByteArray]Int = minted.get_policy(policy);
  reference_assets_map: Map[AssetClass]Int = minted_assets.map_keys((asset_id: ByteArray) -> AssetClass {
    original_asset_name: ByteArray = asset_id.slice(BALLOT_NAME_PREFIX.length, asset_id.length);
    AssetClass::new(REFERENCE_POLICY_HASH, original_asset_name)
  });
  tx_sends_to_self: Bool = outputs.any((tx_out: TxOutput) -> Bool {
    reference_assets_map.all((voting_asset: AssetClass, amount: Int) -> Bool {
      tx_out.value.contains(Value::new(voting_asset, amount))
    })
  });
  if (tx_sends_to_self) {
    true
  } else {
    print("The NFTs with voting power (" + REFERENCE_POLICY_HASH.serialize().show() + ") for the ballots were never sent-to-self");
    false
  }
}

It would take Map[AssetClass]Int as one parameter instead of AssetClass and Int as two.

[christianschmitz]

Ok, will try to implement Value::from_map(Map[MintingPolicyHash]Map[ByteArray]Int) -> Value

[christianschmitz]

Added from_map (v0.7.1)

So you don't need map_keys anymore. You can just do reference_assets: Value = Value::from_map(Map[MintingPolicyHash]Map[ByteArray]Int{ REFERENCE_POLICH_HASH: minted_assets })

I'm curious to see how well value.contains() performs vs. the naive loop.

Another slight improvement could be to use outputs.head.value.contains() if the output containing the voter_id tokens is first (sadly won't be the case when eg. using cardano-cli because it always puts the change-output first)

[thaddeusdiamond]

Wow! So the following code scaled all the way up to 23 at a time:

func assets_were_spent(minted: Value, policy: MintingPolicyHash, outputs: []TxOutput) -> Bool {
  minted_assets: Map[ByteArray]Int = minted.get_policy(policy);
  reference_assets_names: Map[ByteArray]Int = minted_assets.map_keys((asset_id: ByteArray) -> ByteArray {
    asset_id.slice(BALLOT_NAME_PREFIX.length, asset_id.length)
  });
  reference_assets: Map[MintingPolicyHash]Map[ByteArray]Int = Map[MintingPolicyHash]Map[ByteArray]Int {
    REFERENCE_POLICY_HASH: reference_assets_names
  };
  tx_sends_to_self: Bool = outputs.any((tx_out: TxOutput) -> Bool {
    tx_out.value.contains(Value::from_map(reference_assets))
  });
  if (tx_sends_to_self) {
    true
  } else {
    print("The NFTs with voting power (" + REFERENCE_POLICY_HASH.serialize().show() + ") for the ballots were never sent-to-self");
    false
  }
}

Then, if I change it to outputs.head.value.contains() it scales to 26! I'm going to set it to 20 at a time that should be plenty to account for most voters. Thank you for the new APIs! Closing this ticket.

[christianschmitz]

v0.9.0 of Helios has a bunch of code-generation improvements.

You can test it to see if you can scale to more than 26