Difficulties sending a transaction immediately after a fresh account is funded

Issue Description

If you try to send a transaction from a brand-new account immediately after that account is funded, you sometimes get the wrong starting Sequence number, causing your transaction to be rejected by consensus. Often, the preliminary result from submit is tesSUCCESS, before the transaction eventually gets dropped with (presumably) a tefPAST_SEQ result.

In particular, the account_info response is weird before the funding transaction has been validated.

Previously this was not an issue because starting sequence numbers were deterministic, but thanks to deletable accounts this is not the case anymore.

Steps to Reproduce

This gist has the steps to reproduce. Save as an HTML file and open it in your browser. Optionally watch along in the Testnet Explorer. Note, ripple-lib's prepareTransaction() uses account_info to get the right sequence number.

Alternatively, you can do it by hand:

Open the Testnet Faucet page.
Open the WebSocket API tool, switch to the Testnet, and load the account_info request.
Generate Testnet credentials.
Quickly, copy-paste the newly-funded Address into the address field of the WebSocket tool and send the WS Tool request.

Expected Result

A transaction prepared with the Sequence number as reported in the current ledger account_info should succeed most of the time.

Actual Result

The transaction usually gets a tentative result of tesSUCCESS but almost always becomes invalid and never makes it into a consensus ledger. (Aside: This is harder to detect than it should be because of #3727 and #3750.)

Root Cause

With deletable accounts, your starting sequence number is based on the ledger index when your account was first funded. In other words, if the transaction funding your account has not been fully validated, you don't know for certain what your starting sequence number will be. A possible sequence of events is:

The transaction (F) funding your account is proposed for ledger index N.
You prepare and sign your first transaction (T) using Sequence N.
You submit your first transaction (T).
The funding transaction (F) "slips" into ledger index N+1 and is then validated.
Transaction T is now invalid because its Sequence number (N) is lower than your account's current (starting) Sequence number (N+1).

Note, it's not the same type of problem if transaction T is initially proposed for ledger index N and lands above transaction F in the canonical order. In this case, transaction T either gets bumped down later in the canonical order due to the retry rules, or it gets pushed out of ledger N but is still valid and can execute just fine in ledger N+1.

Related Weirdness

If you're fast enough query the account_info (or ledger_entry) very early, the response looks like this:

{
  "id": 2,
  "result": {
    "account_data": {
      "Account": "rpMWR4C6UWc5Smvf5pyPb1AFoXTJYHQa3N",
      "Balance": "1000000000",
      "Flags": 0,
      "LedgerEntryType": "AccountRoot",
      "OwnerCount": 0,
      "PreviousTxnID": "0000000000000000000000000000000000000000000000000000000000000000",
      "PreviousTxnLgrSeq": 0,
      "Sequence": 15702633,
      "index": "25AEBEB84377792DA278529C4E1B22016FEFFB3A6BF90C4536B5271FCD78A6C3"
    },
    "ledger_current_index": 15702633,
    "queue_data": {
      "txn_count": 0
    },
    "validated": false
  },
  "status": "success",
  "type": "response"
}

Notice that the PreviousTxnID and PreviousTxnLgrSeq values are placeholders. You can also see that the Sequence number matches the ledger_current_index, so at least in theory this the correct value for now. That's also borne out by the fact that the preliminary result of the submit is in fact, tesSUCCESS. It's only later that the funding transaction gets pushed back and fails.

Workarounds

The easiest workaround is to wait until the transaction funding your account is validated (usually just a few seconds). So for example:

  while (true) {
    try {
      await api.request("account_info", {account: address, ledger_index: "validated"})
      break
    } catch(e) {
      // you could sleep or something here, but that's awkward to do in JS
    }
  }

Another workaround is basically offline account setup as described in the Deletable Accounts Spec. You create no-op transactions for earlier possible sequence numbers to fill the gaps, then make your first "meaningful" transaction on the highest possible starting sequence number.

Conclusions

I don't think there's going to be a fix for the core problem of, "You don't know your starting Sequence number for sure until the funding transaction is validated."

We might want to look into the account_info response with the placeholders, and possibly return actNotFound instead in some cases.

It would be slightly dangerous to just "guess higher" on the starting Sequence number in account_info if we're not sure, because then the client has potentially created and submitted a transaction that becomes valid for retry after they realize their mistake. That would be OK if the user's response was, "Oh, OK, I'll send a no-op to use up the lower sequence number first," but it seems incredibly likely that people might think, "Oh, I guess I should use a lower Sequence number for this transaction" instead. And that's an express all-expenses-paid trip a "double send" (not the same as a "double spend").

XRPLF / rippled