search

kyuupichan / electrumx

Alternative implementation of spesmilo/electrum-server
Other
723 stars 732 forks source link

blockchain.transaction.get slow #277

Closed cipig closed 6 years ago

cipig commented 6 years ago

It takes up to 10 minutes after a tx was included in a block for blockchain.transaction.get to see the tx. Is this a bug or are there some hidden parameters to tweak this?

kyuupichan commented 6 years ago

I don't think this is anything to do with ElectrumX. Do you have reason to believe that is the case? I also don't think it's true - the TX is available when in the mempool, otherwise it wouldn't show properly in Electrum client.

jl777 commented 6 years ago

I have seen a txid/vout in listunspent return while also the transaction that spends it is known to electrum, ie. in mempool and maybe even history. After a few blocks it all clears up, but it seems that the listunspent is not being updated as fast as needed for coins with 1 minute blocktimes.

also, is the spec that it will be in listunspent even if it is in the mempool return? the gettransaction return is also lagging for fast blocktime coins.

hopefully there is an easy way to speed up the updating of the internal data structures. it is not easy to be scrubbing the false unspents from the listunspent and keeping it all in sync during an atomic swap sequence.

kyuupichan commented 6 years ago

Hmm, the updates are essentially atomic. I'll take a look but I'm skeptical...

kyuupichan commented 6 years ago

If this is a coin with its own weird stuff and handling let me know. I haven't written the altcoin stuff and I don't know if they work reliably. I'm mainly interested in anything that is essentially the same as bitcoin in data structures and operation.

kyuupichan commented 6 years ago

Can you guys also be specific and clear about how you're observing this? For anything to appear in electrum, for example, it has to have requested the tx from the electrum server.

jl777 commented 6 years ago

Not coins specific, I have seen this with bitcoin too.

the listunspent has a txid/vout that is also spent in the mempool list

kyuupichan commented 6 years ago

Electrum doesn't use listunspent in normal operation I believe. How are you using this / what is the client?

jl777 commented 6 years ago

it is a custom application that is using electrumX server.

a tx is submitted to the network (maybe via electrum server maybe via bitcoind) that spends a txid/vout in the listunspent array.

the new tx appears in the mempool, but the listunspent still has the txid/vout that was spent

so I am having to treat the listunspent return as advisory and then do a mempool and history and invalidate the listunspent return. this requires a lot more calls as all the transactions need to be fetched, and all the vins checked to see if the listunspent txid/vouts have been spent or not

kyuupichan commented 6 years ago

Ah, OK, I can perhaps believe listunspent still lists newly-spent outputs. I'll take a look at that.

However transaction.get is simply passed to bitcoind. The only way that could not appear is if the TX was originally submitted to a different bitcoind. I regularly see my bitcoind doesn't have a mempool TX that other nodes have, for whatever reason.

jl777 commented 6 years ago

when the tx is still unconfirmed in the mempool a node that is not a full relay node, will likely not have a transaction outside its wallet. but even for pruned nodes, once it gets into a block, the bitcoind should have the rawtransaction, even for transactions not associated with the wallet.

when it goes outside the pruned depth, then it wont be available. but it sounds like configuring the bitcoind can help a bit with the get transaction issue.

It would really help if the listunspent removed the spends from the mempool! thanks!

jl777 commented 6 years ago

oh, our use case is barterDEX where we are doing cross chain atomic swaps using electrumX. It is working pretty well but for the few issues we have reported.

a fantastic thing, the electrumX!!

kyuupichan commented 6 years ago

I take it "cipig" and "jl777" are "we" in "issues we have reported"?

jl777 commented 6 years ago

yes

kyuupichan commented 6 years ago

I have updated the listunspent methods in 1.1.2 to remove in-mempool spends. I think this resolves the issue you were seeing. If not, please open a new ticket.

jl777 commented 6 years ago

thank you!! we will get this tested and let you know

kyuupichan commented 6 years ago

How did the testing go?

jl777 commented 6 years ago

sorry for the delay, we just regenerated all the DB for the 20 coins and about to test them this weekend

jl777 commented 6 years ago

a purely electrum atomic swap worked! this is not 100% of the test cases, but a very good sign. Things to seem better, so unless we find something, no news is good news