XRPLF / rippled

Decentralized cryptocurrency blockchain daemon implementing the XRP Ledger protocol in C++
https://xrpl.org
ISC License
4.48k stars 1.45k forks source link

RIPD-1847 fix select query condition #4955

Closed oleks-rip closed 3 months ago

oleks-rip commented 3 months ago

High Level Overview of Change

perf: improve account_tx SQL query:

The witness server makes heavily use of the account_tx RPC command. Perf testing showed that the SQL query used by account_tx became unacceptably slow when the DB was large and there was a marker parameter. The plan for the query showed only indexed reads. This appears to be an issue with the internal SQLite optimizer. This patch rewrote the query to use UNION instead of OR and significantly improves performance. See RXI-896 and RIPD-1847 for more details.

Example

was:

SELECT AccountTransactions.LedgerSeq,AccountTransactions.TxnSeq,
  Status,RawTxn,TxnMeta
  FROM AccountTransactions, Transactions WHERE
  (AccountTransactions.TransID = Transactions.TransID AND
  AccountTransactions.Account = 'rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh' AND
  AccountTransactions.LedgerSeq BETWEEN '415398' AND '415441')
  OR
  (AccountTransactions.TransID = Transactions.TransID AND
  AccountTransactions.Account = 'rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh' AND
  AccountTransactions.LedgerSeq = '415442' AND
  AccountTransactions.TxnSeq <= '152')
  ORDER BY AccountTransactions.LedgerSeq DESC,
  AccountTransactions.TxnSeq DESC
  LIMIT 400;

became:

SELECT AccountTransactions.LedgerSeq,AccountTransactions.TxnSeq,Status,RawTxn,TxnMeta
  FROM AccountTransactions, Transactions WHERE
  (AccountTransactions.TransID = Transactions.TransID AND
  AccountTransactions.Account = 'rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh' AND
  AccountTransactions.LedgerSeq BETWEEN 415398 AND 415441)
UNION
SELECT AccountTransactions.LedgerSeq,AccountTransactions.TxnSeq,Status,RawTxn,TxnMeta
  FROM AccountTransactions, Transactions WHERE
  (AccountTransactions.TransID = Transactions.TransID AND
  AccountTransactions.Account = 'rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh' AND
  AccountTransactions.LedgerSeq = 415442 AND
  AccountTransactions.TxnSeq <= 152)
  ORDER BY AccountTransactions.LedgerSeq DESC,
  AccountTransactions.TxnSeq DESC
  LIMIT 400;

Type of Change

codecov-commenter commented 3 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 76.95%. Comparing base (69143d7) to head (9a0c4df).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #4955 +/- ## ======================================== Coverage 76.95% 76.95% ======================================== Files 1127 1127 Lines 131696 131695 -1 Branches 39578 39520 -58 ======================================== + Hits 101341 101345 +4 + Misses 24440 24398 -42 - Partials 5915 5952 +37 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.