[BUG] - `calculate-min-fee` for pre-Conway release gives fees too low

[rphair]

External - Other

Summary cardano-cli transaction calculate-min-fee for version 8.24.0.0 calculates fees too low to submit the signed Tx with that fee to a 8.12.2 node and/or to the chain itself.

Steps to reproduce

1. Prepare & sign a transaction with the latest release version of cardano-cli at this time.
2. Observe that it is several thousand lovelace lower than customary values for version 8.x prepared & submitted successfully.
3. See that Tx is rejected with the usual error AlonzoInBabbageUtxoPredFailure (FeeTooSmallUTxO ...)
4. (optional) Add several thousand lovelace + a little margin for error & see the Tx's get accepted OK.

Expected behavior A returned calculate-min-fee value allowing a successful Tx submission.

System info (please complete the following information):

• OS Name: Ubuntu 22.04
• Observed in latest GitHub binary releases:
  • Node version: 8.12.2
  • CLI version: 8.24.0.0

Additional context Submitted on Cardano Forum as below (Does calculate-min-fee give fees too low for interim node?) with lightweight confirmation from 3 SPO's so far, and on Discord requesting an issue be filed (cc @disassembler), as reported & observed in this epoch 495:

---

Our pool recently took the node upgrade to 8.12.2 which we know removes the Conway genesis & many relevant parameters for the Conway era from the configuration, in anticipation of adding them back in 9.x.

In our only recent use of cardano-cli since the upgrade (actual CLI version = 8.24.0.0), cardano-cli transaction calculate-min-fee has been returning astonishingly lower values: thousands of lovelace lower than needed to accept Tx's on the chain. (We've gotten through this today by averaging fees according to what they were before the upgrade & adding a margin.)

I immediately double-checked that the protocol.json file passed as a parameter was the correct one downloaded for this epoch (edit: 495).

I did see that some arguments have been dropped from calculate-min-fee — including --tx-in-count and --tx-out-count, which we had always used before — although the results are the same whether or not these deprecated arguments are used.

We've had this problem today [edit: UTC 2024-07-06 16:00] with 2 types of Tx's: a transaction that combines some UTxO's at the same address into a single UTxO while withdrawing stake rewards, and a pool re-registration: both calculating fees too low by thousands of lovelace.

Here's the command sequence for the first example, giving us (consistently) the unusually & unusably low estimated fee (of 174653 lovelace compared to at least 180K even for 3 inputs without the rewards withdrawal):

cardano-cli transaction build-raw \
--tx-in UTxO1 \
--tx-in UTxO2 \
--tx-in UTxO3 \
--withdrawal $(cat low/stake.addr)+0 \
--tx-out $(cat low/payment.addr)+0 \
--fee 0 \
--out-file balance.test

cardano-cli transaction calculate-min-fee \
--tx-body-file balance.test  \
--witness-count 2 \
--protocol-params-file protocol.json
174653 # output (generally our fees in the 180K to 183K range)

I've been following developer & SPO threads through this release and the node lead's general advice was that "tooling could be affected on 8.12.x"... but by "tooling" I assumed things like pool installation scripts & not the CLI itself. I know if the fee calculations had changed then it would temporarily break things like the SPO scripts: but shouldn't the raw CLI work the way it always has? ...

---

I remarked when reporting on IOG SPO Discord:

Something about the changed fee costing seems to have made advance fee calculations inaccurate (too low by several thousand lovelace) for ordinary transactions

and wonder if shunting out the Conway parameters has not led to some terms in the fee calculation equation being zeroed out.

[AndrewWestberg]

@rphair The fees were incorrect before. Now, you need to provide more realistic values for the dummy tx you use when calculating min fee. When you set a tx-out value, withdrawal, or a fee value to 0, that doesn't use enough bytes so your calculation ends up too low.

Once you have a realistic fee value, rebuild the tx with the correct fees.

[rphair]

@AndrewWestberg as per our concurrent conversation, noting here for the record that the same fee is calculated by cardano-node 8.24.0.0 when given the same values to be used for our actual transaction, i.e. the "realistic" values & therefore the same byte sizes that should be used for Tx costing.

[disassembler]

The issue is the 0 values for fee and withdrawal. When those are updated to the actual values it increases the size of the transaction, thereby increasing the min fee for the transaction to submit. calculate-min-fee needs to be ran recursively on fee changes because an increase in the fee might increase the tx size by one or more bytes.

[rphair]

Finally, upon further review: despite Tx parameters sometimes not changing the calculated fee, there is generally (as @AndrewWestberg and @disassembler said) a small but crucial difference between the cost of zero sized Tx fields and realistically sized fields; observable in the exact error message when the inadequate Tx was submitted (calculated as originally reported above):

Command failed: transaction submit Error: Error while submitting tx: ShelleyTxValidationError ShelleyBasedEraBabbage (ApplyTxError (UtxowFailure (UtxoFailure (AlonzoInBabbageUtxoPredFailure (FeeTooSmallUTxO (Coin 174653) (Coin 173949)))) :| []))

I expect to verify & that other operators will confirm using operating-range values rather than a placeholding 0 in the dummy Tx for fee calculation will eliminate any such problems.