cculianu / Fulcrum

A fast & nimble SPV Server for BCH, BTC, and LTC
Other
325 stars 73 forks source link

Transaction distance from mempool tip is off, shows much closer than it actually is. #221

Closed nostitos closed 6 months ago

nostitos commented 6 months ago

image

image

image Last one is my self hosted Fullcrum

cculianu commented 6 months ago

Isn’t this more an issue with electrum wallet?

Also isnt btc mempool particularly chaotic now like aren’t all estimates off because fees are changing constantly?

cculianu commented 6 months ago

Also one thing that may be happening: if your btc node just comes up it won’t have an accurate mempool view. It will be missing txns so estimate will be way off.

Is your node always up?

does its mempool txn count resemble that of mempool.space ?

nostitos commented 6 months ago

This number comes from the server connected.

My node has been up for about 6months without any downtimes.

Since the Tx is at 60sat/byte it's still unconfirmed, Electrum now shows 8mb image

Mempool.space doesn't specifically say how close it is in the "queue" but https://jochen-hoenicke.de/queue/#BTC,8h,weight show it as 16mb away. image

mempool shows it as ~15mb away image

cculianu commented 6 months ago

Well how many txns are in your mempool vs normal mempools? Are you doing something like non-standard relay rules or running knots or ordislow or something?

nostitos commented 6 months ago

Well how many txns are in your mempool vs normal mempools? Are you doing something like non-standard relay rules or running knots or ordislow or something?

I'm running the default settings from Parmanode

nostitos commented 6 months ago

Do you know of other public fulcrum server I could compare with, It's very problematic to estimate fees and confirmation ETA with this issue.

cculianu commented 6 months ago

You could use the FulcrumAdmin peers command and pick a server.

Or use this website to find one: https://1209k.com/bitcoin-eye/ele.php?chain=btc

nostitos commented 6 months ago

Quick survey for a tx at 26sat/b

fulcrum.sethforprivacy.com:50002 = 21.43mb Fulcrum 1.9.7 bitcoins.sk:56002 = 21.52mb Fulcrum electrum.jochen-hoenicke.de = 21.98MB Fulcrum My Fulcrum = 22.12mb (mempool set to 3gb)

electrum.bitaroo.net = 38.57mb ElectrumX electrum.emzy.de = 38.62MB electrumX btc.hodler.ninja:50002 = 39.01mb ElextrumX

image

So the discrepancy is between implementations. (Small variation must be caused by looking delays)

Could it be related to byte vs vbytes calculation?

From our experience the ElectrumX calculation is the more correct one. Whenever it's barely below 1mb it's included in next block.

cculianu commented 6 months ago

Yeah you may be onto something. Could very well be bytes vs vbytes thing. Fulcrum was originally written to pretty much ignore segwit stuff. I will look into it. Thanks for doing the investigations.

cculianu commented 6 months ago

I think I have a fix ready -- are you able to build from source and test it? Or shall I prepare a static binary for you? (What platform if so?).

Note that I am travelling and my BTC node just doesn't have a full mempool because it's been offline most of the time and I just fired it up.. but the mempool is mostly empty.. so it's hard to really test, but I can try.

Would appreciate if you can test my fix.

cculianu commented 6 months ago

I believe this is fixed in #225. I closed this issue for now but let me know if you want me to send you a binary to test .. I definitely can see now how it was using raw serialized size, rather than vsize, and thus the calculated fees txns were paying were "off" (too low), so you could end up told by Electrum that you are farther ahead in the queue than you really are given the fee calculation electrum was doing for your own txns.

Anyway I believe it's fixed now.

Let me know if you want to test it! Can send you a binary...

cculianu commented 6 months ago

BTW the fix is now live with released version v1.9.8. If you want, you can connect to servers advertising version 1.9.8 or greater (as of the time of this writing that would be: electrum.jochen-hoenicke.de port 50006).

nostitos commented 6 months ago

Amazing, thank you so much!