Test fails with `TooManyCollateralInputs` when validator generates a lot of thunks

VKFisher commented 2 years ago

Summary

When testing a contract I ran into an issue where transaction validation fails with the following error:

CardanoLedgerValidationError "ApplyTxError [UtxowFailure (WrappedShelleyEraFailure (UtxoFailure (TooManyCollateralInputs 5 8)))]"

After investigation it appears that the issue is caused by computations that generate a lot of thunks during evaluation. I've reproduced the issue in this minimal example:

{-# LANGUAGE MultiParamTypeClasses #-}
{-# LANGUAGE TemplateHaskell #-}

module Contracts.Examples.Collateral where

import Control.Monad (void)
import Ledger (Address, ScriptContext)
import qualified Ledger.Constraints as Constraints
import Ledger.Typed.Scripts as Scripts
import Playground.Contract
import Plutus.Contract
    ( Promise,
      submitTxConstraints,
      submitTxConstraintsSpending,
      collectFromScript,
      selectList )
import Plutus.V1.Ledger.Ada (lovelaceValueOf)
import qualified PlutusTx
import PlutusTx.Prelude hiding (Applicative (..))

-- | These are the data script and redeemer types. We are using an integer
--   value for both, but you should define your own types.
newtype MyDatum = MyDatum Integer deriving newtype (PlutusTx.ToData, PlutusTx.FromData, PlutusTx.UnsafeFromData)

PlutusTx.makeLift ''MyDatum

newtype MyRedeemer = MyRedeemer Integer deriving newtype (PlutusTx.ToData, PlutusTx.FromData, PlutusTx.UnsafeFromData)

PlutusTx.makeLift ''MyRedeemer

{-# INLINEABLE replicate #-}
replicate :: Integer -> a -> [a]
replicate n x = f n []
  where
    f n' acc
      | n' > 0 = f (n' - 1) (x : acc)
      | otherwise = acc

-- | This method is the spending validator (which gets lifted to
--   its on-chain representation).
{- HLINT ignore "Use sum" -}
{-# INLINEABLE validateSpend #-}
validateSpend :: MyDatum -> MyRedeemer -> ScriptContext -> Bool
validateSpend (MyDatum expectedRes) (MyRedeemer input) _ = foldl (+) 0 (replicate input 1) == expectedRes

-- | The address of the contract (the hash of its validator script).
contractAddress :: Address
contractAddress = Scripts.validatorAddress starterInstance

data Starter

instance Scripts.ValidatorTypes Starter where
  type RedeemerType Starter = MyRedeemer
  type DatumType Starter = MyDatum

-- | The script instance is the compiled validator (ready to go onto the chain)
starterInstance :: Scripts.TypedValidator Starter
starterInstance =
  Scripts.mkTypedValidator @Starter
    $$(PlutusTx.compile [||validateSpend||])
    $$(PlutusTx.compile [||wrap||])
  where
    wrap = Scripts.wrapValidator @MyDatum @MyRedeemer

-- | The schema of the contract, with two endpoints.
type Schema =
  Endpoint "publish" (Integer, Integer)
    .\/ Endpoint "redeem" Integer

contract :: AsContractError e => Contract () Schema e ()
contract = selectList [publish, redeem]

-- | The "publish" contract endpoint.
publish :: AsContractError e => Promise () Schema e ()
publish = endpoint @"publish" $ \(expectedRes, lockedFunds) -> do
  let tx = Constraints.mustPayToTheScript (MyDatum expectedRes) (lovelaceValueOf lockedFunds)
  void $ submitTxConstraints starterInstance tx

-- | The "redeem" contract endpoint.
redeem :: AsContractError e => Promise () Schema e ()
redeem = endpoint @"redeem" $ \myRedeemerValue -> do
  unspentOutputs <- utxosAt contractAddress
  let redeemer = MyRedeemer myRedeemerValue
      tx = collectFromScript unspentOutputs redeemer
  void $ submitTxConstraintsSpending starterInstance unspentOutputs tx

endpoints :: AsContractError e => Contract () Schema e ()
endpoints = contract

mkSchemaDefinitions ''Schema

$(mkKnownCurrencies [])

The relevant part is the use of foldl in the validateSpend function. If it is replaced with foldr, the validation passes successfully. This is not an OOM issue, since an identical computation runs fine off-chain. 50000 is not an outrageous amount, either.

Plutus Core is supposed to be strict, but it seems like its emulation is affected by laziness here.

Steps to reproduce the behavior

Run the following test

{-# LANGUAGE DataKinds #-}
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE MultiParamTypeClasses #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE TypeFamilies #-}
{-# LANGUAGE NoImplicitPrelude #-}

module Contracts_test.Collateral_test (tests) where

import Contracts.Examples.Collateral
import Control.Monad hiding (fmap)
import Data.Text (Text)
import Plutus.Contract.Test
  ( assertNoFailedTransactions,
    checkPredicate,
    w1,
    w2,
  )
import Plutus.Trace.Emulator as Emulator
  ( ContractHandle,
    EmulatorTrace,
    activateContractWallet,
  )
import qualified Plutus.Trace.Emulator as Trace
import Test.Tasty (TestTree, testGroup)
import Prelude

publishTrace ::
  ContractHandle () Schema Text ->
  (Integer, Integer) ->
  EmulatorTrace ()
publishTrace hdl (expectedRes, v) = do
  void $ Trace.waitNSlots 1
  Trace.callEndpoint @"publish" hdl (expectedRes, v)
  void $ Trace.waitNSlots 1

redeemTrace ::
  ContractHandle () Schema Text ->
  Integer ->
  EmulatorTrace ()
redeemTrace hdl input = do
  void $ Trace.waitNSlots 1
  Trace.callEndpoint @"redeem" hdl input
  void $ Trace.waitNSlots 1

tests :: TestTree
tests =
  testGroup
    "collateral"
    [ collateralTests
    ]

collateralTests :: TestTree
collateralTests =
  testGroup
    "collateral"
    [ checkPredicate
        "utxo count does not exceed limit"
        assertNoFailedTransactions
        $ do
          h1 <- activateContractWallet w1 endpoints
          h2 <- activateContractWallet w2 endpoints

          publishTrace h1 (50000, 10000000)
          redeemTrace h2 50000
    ]

OR paste the contract code into playground and simulate the transaction from the test

Actual Result

Transaction validation fails with the following error:

        [INFO] Slot 4: 00000000-0000-4000-8000-000000000001 {Wallet W[2]}:
                         Receive endpoint call on 'redeem' for Object (fromList [("contents",Array [Object (fromList [("getEndpointDescription",String "redeem")]),Object (fromList [("unEndpointValue",Number 45000.0)])]),("tag",String "ExposeEndpointResp")])
        [INFO] Slot 4: W[2]: Balancing an unbalanced transaction:
                               Tx:
                                 Tx f0199c14819d959db8cd8aa80a80855bd4c8a29dc36d1e92d267eed9ce6fee26:
                                   {inputs:
                                      - 29828087b51af5d4fc54c1beb568cd95935dcea5e07dbbbeb323b63c9bf88009!1
                                        45000
                                   collateral inputs:
                                   outputs:
                                   mint: Value (Map [])
                                   fee: Value (Map [])
                                   mps:
                                   signatures:
                                   validity range: Interval {ivFrom = LowerBound NegInf True, ivTo = UpperBound PosInf True}
                                   data:}
                               Requires signatures:
                               Utxo index:
                                 ( 29828087b51af5d4fc54c1beb568cd95935dcea5e07dbbbeb323b63c9bf88009!1
                                 , - Value (Map [(,Map [("",10000000)])]) addressed to
                                     ScriptCredential: 4f27b58b078a333f5a907075743c6e55af456f347e13fea47fc602e9 (no staking credential) )
                               Validity range:
                                 (-∞ , +∞)
        [INFO] Slot 4: W[2]: Finished balancing:
                               Tx 04730ff5bfcb7c108ddf8d3d6e1e78c5a910c597e768c94194ac6ee6915fa6d9:
                                 {inputs:
                                    - 29828087b51af5d4fc54c1beb568cd95935dcea5e07dbbbeb323b63c9bf88009!1
                                      45000
                                    - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!20

                                    - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!21

                                    - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!22

                                    - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!23

                                    - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!24

                                 collateral inputs:
                                   - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!20

                                   - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!21

                                   - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!22

                                   - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!23

                                   - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!24

                                   - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!25

                                 outputs:
                                   - Value (Map [(,Map [("",8988447)])]) addressed to
                                     PubKeyCredential: 80a4f45b56b88d1139da23bc4c3c75ec6d32943c087f250b86193ca7 (no staking credential)
                                 mint: Value (Map [])
                                 fee: Value (Map [(,Map [("",51011553)])])
                                 mps:
                                 signatures:
                                 validity range: Interval {ivFrom = LowerBound NegInf True, ivTo = UpperBound PosInf True}
                                 data:}
        [INFO] Slot 4: W[2]: Signing tx: 04730ff5bfcb7c108ddf8d3d6e1e78c5a910c597e768c94194ac6ee6915fa6d9
        [INFO] Slot 4: W[2]: Submitting tx: 04730ff5bfcb7c108ddf8d3d6e1e78c5a910c597e768c94194ac6ee6915fa6d9
        [INFO] Slot 4: W[2]: TxSubmit: 04730ff5bfcb7c108ddf8d3d6e1e78c5a910c597e768c94194ac6ee6915fa6d9
        [INFO] Slot 4: 00000000-0000-4000-8000-000000000001 {Wallet W[2]}:
                         Contract instance stopped (no errors)
        [WARNING] Slot 4: TxnValidationFail Phase1 04730ff5bfcb7c108ddf8d3d6e1e78c5a910c597e768c94194ac6ee6915fa6d9: CardanoLedgerValidationError "ApplyTxError [UtxowFailure (WrappedShelleyEraFailure (UtxoFailure (TooManyCollateralInputs 5 6)))]"
          src/Plutus/Contract/Test.hs:300:
          utxo count does not exceed limit

Expected Result

Transaction validates successfully

Describe the approach you would take to fix this

No response

System info

OS: Ubuntu 20.04 Plutus: v2022-04-06

nc6 commented 2 years ago

The error you're seeing is not a Plutus error, it's a ledger phase-1 error, and will result in the script not even running. The error occurs because you have 6 collateral inputs where you're only allowed to have a maximum of 5.

My guess is that this is an issue in the balancing code. Probably the switch from foldr tofoldl` increases the computed costs such that the system needs extra collateral, and it hasn't been coded to bound the number of inputs at 5.

Try making sure you have larger inputs available to use as collateral, perhaps by consolidating some inputs.

VKFisher commented 2 years ago

@nc6

Here's the transaction before balancing, with a single input.

Tx:
   Tx f0199c14819d959db8cd8aa80a80855bd4c8a29dc36d1e92d267eed9ce6fee26:
     {inputs:
        - 29828087b51af5d4fc54c1beb568cd95935dcea5e07dbbbeb323b63c9bf88009!1
          45000
     collateral inputs:
     outputs:
     mint: Value (Map [])
     fee: Value (Map [])
     mps:
     signatures:
     validity range: Interval {ivFrom = LowerBound NegInf True, ivTo = UpperBound PosInf True}
     data:}
 Requires signatures:
 Utxo index:
   ( 29828087b51af5d4fc54c1beb568cd95935dcea5e07dbbbeb323b63c9bf88009!1
   , - Value (Map [(,Map [("",10000000)])]) addressed to
       ScriptCredential: 4f27b58b078a333f5a907075743c6e55af456f347e13fea47fc602e9 (no staking credential) )
 Validity range:
   (-∞ , +∞)

Here's that same transaction after balancing. There are six collateral inputs with the identifier ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958 which is different from the original input and doesn't appear anywhere in the initial transaction.

 Tx 04730ff5bfcb7c108ddf8d3d6e1e78c5a910c597e768c94194ac6ee6915fa6d9:
   {inputs:
      - 29828087b51af5d4fc54c1beb568cd95935dcea5e07dbbbeb323b63c9bf88009!1
        45000
      - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!20

      - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!21

      - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!22

      - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!23

      - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!24

   collateral inputs:
     - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!20

     - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!21

     - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!22

     - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!23

     - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!24

     - ef0ca0fb043642529818003be5a6cac88aac499e4f8f1cbc3bdb35db2b7f6958!25

   outputs:
     - Value (Map [(,Map [("",8988447)])]) addressed to
       PubKeyCredential: 80a4f45b56b88d1139da23bc4c3c75ec6d32943c087f250b86193ca7 (no staking credential)
   mint: Value (Map [])
   fee: Value (Map [(,Map [("",51011553)])])
   mps:
   signatures:
   validity range: Interval {ivFrom = LowerBound NegInf True, ivTo = UpperBound PosInf True}
   data:}

I'm assuming that all the inputs were produced by the emulator. Since I didn't specify any of these, I'm not sure how to go about consolidating them. Are you saying I should make an unbalanced transaction and then modify it?

IntersectMBO / plutus-apps