My node does not catch all pending transactions.

[coozebra]

I've recently upgraded geth from v1.3.13 to v1.4.11 and the node does not catch all pending transactions. Here is config.toml.

` [Eth] NetworkId = 56 LightPeers = 100 TrieTimeout = 150000000000

[Eth.Miner] GasCeil = 140000000 GasPrice = 3000000000 Recommit = 10000000000

[Eth.TxPool] Locals = [] NoLocals = true Journal = "transactions.rlp" Rejournal = 3600000000000 PriceLimit = 3000000000 PriceBump = 10 AccountSlots = 200 GlobalSlots = 8000 AccountQueue = 200 GlobalQueue = 4000

[Eth.GPO] Blocks = 20 Percentile = 60 OracleThreshold = 1000

[Node] IPCPath = "geth.ipc" HTTPHost = "localhost" InsecureUnlockAllowed = false HTTPPort = 8547 HTTPVirtualHosts = ["localhost"] HTTPModules = ["eth", "net", "web3", "txpool", "parlia"] WSPort = 8548 WSModules = ["net", "web3", "eth"]

[Node.P2P] MaxPeers = 300 NoDiscovery = false StaticNodes = [] ListenAddr = ":30311" EnableMsgEvents = false

[Node.LogConfig] FilePath = "bsc.log" MaxBytesSize = 10485760 Level = "info" FileRoot = "" `

Please help me resolve an issue. Thank you!

[STdevK]

Please share node start-up command and the node sync log

[coozebra]

startup command is: ./geth --config ./config.toml --datadir ./bscdata --ipcpath ./bscdata/geth.ipc --cache 18000 --history.transactions=90000 --syncmode=full --db.engine=pebble --tries-verify-mode=none --ws --ws.addr 0.0.0.0 --ws.origins '*' --ws.api eth,net,web3,txpool,debug

node log is: t=2024-07-09T06:11:03+0200 lvl=info msg="Imported new chain segment" number=40312199 hash=0xa04b720f3a0b690be96975f0a0f6c71b3ecf378cc0c9b56032904cc0a402acd2 miner=0x978F05CED39A4EaFa6E8FD045Fe2dd6Da836c7DF blocks=1 txs=80 blobs=0 mgas=10.772564 elapsed=53.978ms mgasps=199.57195427959203 snapdiffs="7.22 MiB" triedirty="0.00 B" trieimutabledirty="0.00 B" t=2024-07-09T06:11:06+0200 lvl=info msg="Imported new chain segment" number=40312200 hash=0x154fabb4d5edb239fdcad75d434d1cfb2b4003eea06f1cab52e3bb8b49859af5 miner=0x9f1b7FAE54BE07F4FEE34Eb1aaCb39A1F7B6FC92 blocks=1 txs=99 blobs=0 mgas=10.881358 elapsed=64.818ms mgasps=167.87386803610374 snapdiffs="7.24 MiB" triedirty="0.00 B" trieimutabledirty="0.00 B" t=2024-07-09T06:11:09+0200 lvl=info msg="Imported new chain segment" number=40312201 hash=0x49504343c5780f342b291e83e13237c96dec36c8f6da75c7f1cc05582e88aec2 miner=0xB4647b856CB9C3856d559C885Bed8B43e0846a47 blocks=1 txs=99 blobs=0 mgas=11.351293 elapsed=54.148ms mgasps=209.63173008566127 snapdiffs="7.27 MiB" triedirty="0.00 B" trieimutabledirty="0.00 B" t=2024-07-09T06:11:12+0200 lvl=info msg="Imported new chain segment" number=40312202 hash=0xae04209ad94f4a0e5dd4d28168a15a42a648905e1ba518608301feabe31933a3 miner=0xB997Bf1E3b96919fBA592c1F61CE507E165Ec030 blocks=1 txs=77 blobs=0 mgas=6.260272 elapsed=29.581ms mgasps=211.6247453218681 snapdiffs="7.29 MiB" triedirty="0.00 B" trieimutabledirty="0.00 B" t=2024-07-09T06:11:12+0200 lvl=error msg="Cannot prune blob ancient" block=40312202 expectTail=39759114 err="the input tail&head is less than offset" t=2024-07-09T06:11:15+0200 lvl=info msg="Imported new chain segment" number=40312203 hash=0x39ecd3b68832bafb78499ab929535f18a542de5366229f9fe366ff368f3e08a6 miner=0xbdcc079BBb23C1D9a6F36AA31309676C258aBAC7 blocks=1 txs=67 blobs=0 mgas=5.282943 elapsed=24.651ms mgasps=214.30760251340138 snapdiffs="7.30 MiB" triedirty="0.00 B" trieimutabledirty="0.00 B" t=2024-07-09T06:11:18+0200 lvl=info msg="Imported new chain segment" number=40312204 hash=0x04e4e4d489c38625b6ee7e24a53b9349db3411f72459d104fadd93061f59a4b8 miner=0xCa503a7eD99eca485da2E875aedf7758472c378C blocks=1 txs=86 blobs=0 mgas=7.857192 elapsed=39.470ms mgasps=199.0629600463268 snapdiffs="7.32 MiB" triedirty="0.00 B" trieimutabledirty="0.00 B" t=2024-07-09T06:11:22+0200 lvl=info msg="Imported new chain segment" number=40312205 hash=0x739990c5c6c12cdb8f562f70545c99533aadbbe0df79f8c9af1a1794dba1a327 miner=0xCcB42A9b8d6C46468900527Bc741938E78AB4577 blocks=1 txs=164 blobs=0 mgas=22.617938 elapsed=79.351ms mgasps=285.03375374478014 snapdiffs="7.39 MiB" triedirty="0.00 B" trieimutabledirty="0.00 B" t=2024-07-09T06:11:24+0200 lvl=info msg="Imported new chain segment" number=40312206 hash=0x79b84d865a5c712d00d0f4a6b4cc71a60ebc88f477c5a92ea9da9fa2ce7d86f3 miner=0xD3b0d838cCCEAe7ebF1781D11D1bB741DB7Fe1A7 blocks=1 txs=72 blobs=0 mgas=9.314797 elapsed=38.877ms mgasps=239.59113929188433 snapdiffs="7.41 MiB" triedirty="0.00 B" trieimutabledirty="0.00 B" t=2024-07-09T06:11:27+0200 lvl=info msg="Imported new chain segment" number=40312207 hash=0xc2ccd5eb7e6b7ae448e3c9c226a49a29bf6efd72f93192fa4b6fd29db9ba049a miner=0xF8de5e61322302b2c6e0a525cC842F10332811bf blocks=1 txs=111 blobs=0 mgas=9.241009 elapsed=34.583ms mgasps=267.21225026298305 snapdiffs="7.43 MiB" triedirty="0.00 B" trieimutabledirty="0.00 B" t=2024-07-09T06:11:30+0200 lvl=info msg="Imported new chain segment" number=40312208 hash=0x511b9d5317f43ae8edcef7f988f97d01c3e8a8cb764117b846ccaf160e7f1c15 miner=0xf9814D93b4d904AaA855cBD4266D6Eb0Ec1Aa478 blocks=1 txs=107 blobs=0 mgas=9.903411 elapsed=40.025ms mgasps=247.42915338993978 snapdiffs="7.46 MiB" triedirty="0.00 B" trieimutabledirty="0.00 B" t=2024-07-09T06:11:33+0200 lvl=info msg="Imported new chain segment" number=40312209 hash=0x010f987c5964870c4905020c857ed13af4363dd5596b4c3fd78a515d0072aa98 miner=0x1cFDBd2dFf70C6e2e30df5012726F87731F38164 blocks=1 txs=67 blobs=0 mgas=5.753602 elapsed=28.146ms mgasps=204.4128736437481 snapdiffs="7.47 MiB" triedirty="0.00 B" trieimutabledirty="0.00 B" t=2024-07-09T06:11:36+0200 lvl=info msg="Imported new chain segment" number=40312210 hash=0x596615a6a0176aac7bc842f4ff06a814a2f87522b88e9f980b48ce56b2c38cac miner=0x37e9627A91DD13e453246856D58797Ad6583D762 blocks=1 txs=93 blobs=0 mgas=11.993276 elapsed=55.826ms mgasps=214.83303703189938 snapdiffs="7.51 MiB" triedirty="0.00 B" trieimutabledirty="0.00 B"

[STdevK]

What're your node hardware specification?

[zzzckck]

does not catch all pending transactions.

what does it mean? catch all pending transaction?

[coozebra]

What're your node hardware specification?

Ryzen 9 3900 12 cores | 24 threads RAM 128GB NVME SSD 2*2TB

[coozebra]

does not catch all pending transactions.

what does it mean? catch all pending transaction?

I hope to catch all pending transactions in the mempool. My node worked well with geth v1.3.13 and does not work well with updated geth version v1.4.11. (I noticed geth version update was mandatory) My node catches around 20% of pending transactions now.

[zzzckck]

does not catch all pending transactions.

what does it mean? catch all pending transaction?

I hope to catch all pending transactions in the mempool. My node worked well with geth v1.3.13 and does not work well with updated geth version v1.4.11. (I noticed geth version update was mandatory) My node catches around 20% of pending transactions now.

how do you know the percentage of the transactions your node catch up? by comparing the transaction in blocks with the transactions in your txpool? In this case, it could be caused by MEV, many transactions are send to builders rather than public txpool, which means these transactions are invisible to your node now.

[coozebra]

I have a testing script to check number of pending transactions. The script just logs out number of pending txs every 3 seconds. The number of txs dropped after i upgraded geth. I am not sure all those normal txs went into shady area out of public txpool in 20 days.

[zzzckck]

I have a testing script to check number of pending transactions. The script just logs out number of pending txs every 3 seconds. The number of txs dropped after i upgraded geth. I am not sure all those normal txs went into shady area out of public txpool in 20 days.

there do have some changes regarding to the TxPool's transaction broadcast after v1.4.x. Not sure if it is the root cause for your case. we can do some investigation first.

[coozebra]

Okay, that sounds helpful! What could be the root cause then? I am checking geth code now...

[zhk101]

Your PriceLimit and GasPrice are still using the old 3000000000 min gas value. Try with 1000000000

[coozebra]

Do i need to configure priceLimit and gasPrice to catch pending transactions? I am catching in the following way: provider.on("pending", async (tx) => {...})

[zzzckck]

Your PriceLimit and GasPrice are still using the old 3000000000 min gas value. Try with 1000000000

yes, it could be the problem, @coozebra pls try first.

[zhk101]

Do i need to configure priceLimit and gasPrice to catch pending transactions? I am catching in the following way:

As far as I understand the 'PriceLimit' controls the minimum gas price a pending transaction will have to have for it to be accepted in your node tx pool. If the price is lower, it won't appear in your tx pool. A few month back, they dropped the min price from 3 gwei to 1 gwei and I did notice a big drop in pending transactions in my node (as in, more than half drop). The other setting controls the min price per transaction that your node will accept on submission

[coozebra]

Do i need to configure priceLimit and gasPrice to catch pending transactions? I am catching in the following way:

As far as I understand the 'PriceLimit' controls the minimum gas price a pending transaction will have to have for it to be accepted in your node tx pool. If the price is lower, it won't appear in your tx pool. A few month back, they dropped the min price from 3 gwei to 1 gwei and I did notice a big drop in pending transactions in my node (as in, more than half drop). The other setting controls the min price per transaction that your node will accept on submission

Okay, that is a good explanation. I am resyncing the node with updated config. Lets see what happens.

[coozebra]

Thank you for your support @zzzckck @zhk101 ! My node catches all pending transactions now.