Confirmed Transactions Not Arriving on Sparrow

[asprw]

Hi,

Elaborating further, steps followed are below;

• I connected sparrow to my btc core node via btc proxy service from embassy (start9), and it's connected to the current block height.
• Confirmed that btc core node is up to date and running fine along with btc proxy service.
• I then generated a new wallet on sparrow, added a password to decrypt the wallet, and a passphrase.
• The wallet is a BIP139 wallet.
• I sent SATS to a few addresses from my newly generated wallet.
• Refreshed the wallet. Also set the birth date of the wallet to the date prior to the transfer date.

It's been more than a day of restarting, refreshing and waiting. The wallet connects to the node and shows 'scanning' or 'Finding transactions for ../0/0' when I click refresh but with no progress after waiting for hours. The green toggle button seems to fade in and out like it's 'communicating' with the node?

Pls assist?

Appreciate your time!

Cheers

[craigraw]

The green toggle button seems to fade in and out like it's 'communicating' with the node?

If this is happening, Sparrow is either still trying to connect to Bitcoin Core, or Bitcoin Core is scanning for transactions. You might able to learn more by looking at the Bitcoin Core debug.log. You can also take a look at Sparrow's log (Help menu) to see what might be going wrong. Note Taproot wallets are not currently supported.

[asprw]

Thanks for the response.

On sharing the said logs - from btc core, btc proxy server and Sparrow, with support at Start9 Labs, the embassy seems to be working fine and it looks like the built in BWT from Sparrow wallet is failing, which I'm not able to figure out.

I can confirm that I'm not using a taproot wallet.

Unsure how to proceed...

[craigraw]

Can you share part of the Sparrow log which indicates where BWT is failing?

[asprw]

Thanks for the response. Pls find below. Pls let me know if I should share more. Cheers.

2022-03-20 17:51:56,461 ERROR [JavaFX Application Thread] c.s.s.n.ElectrumServer [null:-1] Failed to start BWT dev.bwt.libbwt.BwtException: JSON-RPC error: transport error: Couldn't connect to host: TTL expired at com.sparrowwallet.merged.module@1.6.2/dev.bwt.libbwt.daemon.NativeBwtDaemon.start(Native Method) at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.Bwt.start(Unknown Source) at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.Bwt.start(Unknown Source) at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.Bwt$ConnectionService$1.call(Unknown Source) at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.Bwt$ConnectionService$1.call(Unknown Source) at javafx.graphics/javafx.concurrent.Task$TaskCallable.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at javafx.graphics/javafx.concurrent.Service.lambda$executeTask$6(Unknown Source) at java.base/java.security.AccessController.doPrivileged(Unknown Source) at javafx.graphics/javafx.concurrent.Service.lambda$executeTask$7(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) 2022-03-20 17:52:25,465 WARN [Thread-34] c.s.s.n.ExchangeSource [null:-1] Error retrieving currency rates (SOCKS: TTL expired) 2022-03-20 18:05:55,481 ERROR [ElectrumServerReadThread] c.s.s.n.TcpTransport [null:-1] Error opening socket inputstream java.net.SocketException: Socket is closed at java.base/java.net.Socket.getInputStream(Unknown Source) at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.TcpTransport.readInputLoop(Unknown Source) at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.ElectrumServer$ReadRunnable.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source)

[asprw]

Some additional bits, if it helps analyze the issue.

2022-03-22 18:56:42,502 WARN [Thread-395] c.s.s.n.ExchangeSource [null:-1] Error retrieving currency rates (Server returned HTTP response code: 403 for URL: https://api.coingecko.com/api/v3/exchange_rates)

And the last bit of the log...

2022-03-22 19:23:53,318 ERROR [ElectrumServerReadThread] c.s.s.n.TcpTransport [null:-1] Error opening socket inputstream java.net.SocketException: Socket is closed at java.base/java.net.Socket.getInputStream(Unknown Source) at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.TcpTransport.readInputLoop(Unknown Source) at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.ElectrumServer$ReadRunnable.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source)

[craigraw]

Thanks for the logs - the key part is this: Couldn't connect to host: TTL expired. This is BWT saying that it can't connect to Bitcoin Core for some reason.

Are you able to get the connection to work in the Server Preferences using the Test Connection button? You should see output from BWT with various details from the node.

[craigraw]

That's all? You don't see this?

[asprw]

Nope. Just the text shared above, on testing connection with embassy node.

On Mar 27, 2022, at 2:10 PM, craigraw @.***> wrote:

That's all? You don't see this?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

[asprw]

Btw, this is the guide followed to connect Sparrow with the Embassy node; https://github.com/Start9Labs/bitcoind-wrapper/blob/master/docs/integrations/sparrow/guide.md

[kn0wmad]

There is something that we have been unable to track down yet here as most of our users connect without issue, but a non-trivial set of users get dodgy connections. This does not typically present in a BWT error however. We are currently testing electrs and it seems better still, but again, this problem only presents with some, not most, users. Perhaps user network setups need to be investigated.

At least in this case, the Bitcoin node is taking calls from Sparrow, so the issue seems to fall on the BWT instance. I'm not sure why it would fail to start up in this case.

[asprw]

Thanks for the message. Well if there’s anything I could/ should be doing from my end, pls let me know as I’m unsure how to approach network investigation in order to solve connectivity issues with Sparrow. Cheers

On Mar 27, 2022, at 9:23 PM, kn0wmad @.***> wrote:

There is something that we have been unable to track down yet here as most of our users connect without issue, but a non-trivial set of users get dodgy connections. This does not typically present in a BWT error however. We are currently testing electrs and it seems better still, but again, this problem only presents with some, not most, users. Perhaps user network setups need to be investigated.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

[craigraw]

If you are not seeing the display in the screenshot above, BWT is definitely not connecting properly.

What you can do is to enable debug logging on startup -l DEBUG - see the FAQ for how do this. The log file can be accessed from the Help menu. There will be a lot of logging, so you will need to search for bwt to find out what is happening when you do the Test Connection. There may be nothing to learn given this is a connection error, but maybe those logs will point to something else.

[asprw]

Thanks, Craigraw.

So I tried starting Sparrow multiple times over the weekend, and one of them yesterday did manage to not only establish connection but also load all the incoming transactions. So am I now able to see the UTXOs with Sats that I couldn't before. But again, the connection went back to being wonky.

I enabled debug logging and ran test connections, and on one of the times, I was also able to see the graphic you shared above. However, it isn't consistent. Here's some of the errors I see that may make sense to you?

DEBUG bitcoincore_rpc > JSON-RPC failed parsing reply of getblockhash: JsonRpc(Transport(SocketError(Custom { kind: Other, error: "TTL expired" }))) WARN bwt > failed syncing with bitcoind: JSON-RPC error: transport error: Couldn't connect to host: TTL expired TRACE bwt::electrum > stop accepting new RPCs TRACE bwt::electrum > closing 1 RPC connections TRACE bwt::electrum > waiting for 1 RPC threads DEBUG bitcoincore_rpc > JSON-RPC failed parsing reply of getrawmempool: JsonRpc(Transport(SocketError(Custom { kind: Other, error: "TTL expired" }))) WARN bwt::electrum > rpc #12 mempool.get_fee_histogram failed: JSON-RPC error: transport error: Couldn't connect to host: TTL expired TRACE bwt::electrum > rpc #13 <- mempool.get_fee_histogram [] DEBUG bitcoincore_rpc > JSON-RPC request: getrawmempool [true] 2022-03-28 12:44:45,115 DEBUG TLSHandshake: api.coingecko.com:443, TLSv1.3, TLS_AES_256_GCM_SHA384, 1302666418 2022-03-28 12:44:50,837 DEBUG ValidationChain: 1521974916, 544235253, 2033425893 2022-03-28 12:44:50,846 DEBUG TLSHandshake: www.sparrowwallet.com:443, TLSv1.3, TLS_AES_256_GCM_SHA384, 2033425893 2022-03-28 12:45:40,680 DEBUG Connection failed com.sparrowwallet.sparrow.net.ServerConfigException: com.sparrowwallet.sparrow.net.ServerConfigException: Could not connect to Bitcoin Core RPC at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.ElectrumServer.getTransport(Unknown Source) at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.ElectrumServer.connect(Unknown Source) at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.ElectrumServer$ConnectionService$1.call(Unknown Source) at com.sparrowwallet.sparrow@1.6.2/com.sparrowwallet.sparrow.net.ElectrumServer$ConnectionService$1.call(Unknown Source) at javafx.graphics/javafx.concurrent.Task$TaskCallable.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at javafx.graphics/javafx.concurrent.Service.lambda$executeTask$6(Unknown Source) at java.base/java.security.AccessController.doPrivileged(Unknown Source) at javafx.graphics/javafx.concurrent.Service.lambda$executeTask$7(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.sparrowwallet.sparrow.net.ServerConfigException: Could not connect to Bitcoin Core RPC ... 12 common frames omitted DEBUG bwt_jni > shutdown signal received WARN jni::wrapper::objects::global_ref > Dropping a GlobalRef in a detached thread. Fix your code if this message appears frequently (see the GlobalRef docs). DEBUG bitcoincore_rpc > JSON-RPC failed parsing reply of getnetworkinfo: JsonRpc(Transport(SocketError(Custom { kind: Other, error: "TTL expired" }))) WARN bwt_jni > JSON-RPC error: transport error: Couldn't connect to host: TTL expired 2022-03-28 12:45:59,677 ERROR Failed to start BWT

[craigraw]

My best guess is that it is something related to the proxy, since I have never encountered this issue with direct connections to Bitcoin Core. SOCKS proxies do send TTL expired errors under certain conditions.

[asprw]

Hi @craigraw, Congrats on winning the award at BanB2022. Well deserved. Just wanted to give you an update that I tried using the wallet directly with the node and not the proxy. Didn't work. Since then, I downloaded the new update and tried connecting to the proxy/node, and now it's doesn't connect at all. Unsure why this is the case. Perhaps my router? ISP? No clue here. Cheers

[craigraw]

Could be the router. Can you ping the node? Check the firewall settings on both sides to make sure the relevant ports are open on your local network.

[tenyo]

Hi, just wanted to add here that I ran into a similar issue using v1.6.5.

• New Native Segwit wallet
• Connecting to Bitcoin Core v22 (myNode) on the same local network, no firewall, no proxy
• Sent 1 TX to my new Sparrow wallet and tried to scan for transactions

When I first configured the connection from Sparrow to my Bitcoin node it worked fine and showed me the bwt banner and a bunch of information about the node. But when I went to the Transactions tab and clicked the toggle at the bottom to connect to my node, it started scanning and got stuck at 84% (I set a birthday for the wallet to the day before to limit the scan). I turned it off/on several times but each time it would get stuck between 80% and 99% (and the toggle was fading in and out indicating it hadn't connected). There was nothing written to the log file. Then after about a dozen tries it finally worked and finished loading 100% and showed me my transaction.

[tenyo]

Adding an update that the connection to to my Bitcoin node is still periodically getting stuck. I can usually reproduce this by toggling the switch off and then on again - it gets stuck at Scanning indefinitely. I was able to capture some relevant logs by running Sparrow in debug mode when trying to enable the connection. You can see at the end it tries to fetch transactions for that block on and on.

 DEBUG bwt::indexer                      > starting initial sync
 DEBUG bwt::wallet                       > checking previous imports
 DEBUG bitcoincore_rpc                   > JSON-RPC request: listlabels []
 TRACE bwt::wallet                       > wallet 0yhlxu5z was imported up to index 349
 TRACE bwt::wallet                       > wallet 7sj5mf96 was imported up to index 350
 DEBUG bitcoincore_rpc                   > JSON-RPC request: getaddressesbylabel ["bwt"]
 DEBUG bitcoincore_rpc                   > JSON-RPC error for getaddressesbylabel: RpcError { code: -11, message: "No addresses with label bwt", data: None }
 TRACE bwt::wallet                       > found 0 previously imported standalone addresses
 DEBUG bitcoincore_rpc                   > JSON-RPC request: listsinceblock [null,1,true,true]
 DEBUG bitcoincore_rpc                   > JSON-RPC request: getblockheader ["00000000000000000005481f63cac68d8c5be10cf5b51480e57a187d1a16df7a",true]
2022-06-26 14:37:21,059 DEBUG  TLSHandshake: api.coingecko.com:443, TLSv1.3, TLS_AES_256_GCM_SHA384, 1302666418
 TRACE bwt::indexer                      > fetched 2 transactions between blocks 0 and 742456
<< Removed some TRACE logs here >>
 INFO  bwt::indexer                      > completed initial sync in 25.916979ms up to height 742456 (total 2 transactions and 2 addresses)
 INFO  bwt::electrum                     > Electrum RPC server running on 127.0.0.1:39487:t (protocol 1.4)
 WARN  jni::wrapper::objects::global_ref > Dropping a GlobalRef in a detached thread. Fix your code if this message appears frequently (see the GlobalRef docs).
2022-06-26 14:37:21,072 DEBUG Electrum ready
2022-06-26 14:37:21,078 DEBUG Bwt ready
 DEBUG bwt                               > starting sync loop
 DEBUG bitcoincore_rpc                   > JSON-RPC request: getblockhash [742456]
 DEBUG bitcoincore_rpc                   > JSON-RPC request: listsinceblock ["00000000000000000005481f63cac68d8c5be10cf5b51480e57a187d1a16df7a",1,true,true]
 TRACE bwt::indexer                      > fetched 0 transactions between blocks 742456 and 742456
 TRACE bwt::real-time                    > debounce sync thread shutting down
 INFO  bwt::electrum                     > [127.0.0.1:54808] connected peer
 DEBUG bwt::electrum                     > rpc #1 <- server.version ["Sparrow","1.4"]
 TRACE bwt::electrum                     > rpc #1 -> server.version ["bwt v0.2.4","1.4"]
 DEBUG bitcoincore_rpc                   > JSON-RPC request: getblockhash [742456]
 DEBUG bitcoincore_rpc                   > JSON-RPC request: listsinceblock ["00000000000000000005481f63cac68d8c5be10cf5b51480e57a187d1a16df7a",1,true,true]
 TRACE bwt::indexer                      > fetched 0 transactions between blocks 742456 and 742456
 DEBUG bitcoincore_rpc                   > JSON-RPC request: getblockhash [742456]
 DEBUG bitcoincore_rpc                   > JSON-RPC request: listsinceblock ["00000000000000000005481f63cac68d8c5be10cf5b51480e57a187d1a16df7a",1,true,true]
 TRACE bwt::indexer                      > fetched 0 transactions between blocks 742456 and 742456
2022-06-26 14:37:31,164 DEBUG  TLSHandshake: www.sparrowwallet.com:443, TLSv1.3, TLS_AES_256_GCM_SHA384, 884178230
 DEBUG bitcoincore_rpc                   > JSON-RPC request: getblockhash [742456]
 DEBUG bitcoincore_rpc                   > JSON-RPC request: listsinceblock ["00000000000000000005481f63cac68d8c5be10cf5b51480e57a187d1a16df7a",1,true,true]
 TRACE bwt::indexer                      > fetched 0 transactions between blocks 742456 and 742456
 DEBUG bitcoincore_rpc                   > JSON-RPC request: getblockhash [742456]
 DEBUG bitcoincore_rpc                   > JSON-RPC request: listsinceblock ["00000000000000000005481f63cac68d8c5be10cf5b51480e57a187d1a16df7a",1,true,true]
 TRACE bwt::indexer                      > fetched 0 transactions between blocks 742456 and 742456
 DEBUG bitcoincore_rpc                   > JSON-RPC request: getblockhash [742456]
 DEBUG bitcoincore_rpc                   > JSON-RPC request: listsinceblock ["00000000000000000005481f63cac68d8c5be10cf5b51480e57a187d1a16df7a",1,true,true]
 TRACE bwt::indexer                      > fetched 0 transactions between blocks 742456 and 742456
 DEBUG bitcoincore_rpc                   > JSON-RPC request: getblockhash [742456]
 DEBUG bitcoincore_rpc                   > JSON-RPC request: listsinceblock ["00000000000000000005481f63cac68d8c5be10cf5b51480e57a187d1a16df7a",1,true,true]
 TRACE bwt::indexer                      > fetched 0 transactions between blocks 742456 and 742456
 DEBUG bitcoincore_rpc                   > JSON-RPC request: getblockhash [742456]
 DEBUG bitcoincore_rpc                   > JSON-RPC request: listsinceblock ["00000000000000000005481f63cac68d8c5be10cf5b51480e57a187d1a16df7a",1,true,true]
 TRACE bwt::indexer                      > fetched 0 transactions between blocks 742456 and 742456
 DEBUG bitcoincore_rpc                   > JSON-RPC request: getblockhash [742456]
 DEBUG bitcoincore_rpc                   > JSON-RPC request: listsinceblock ["00000000000000000005481f63cac68d8c5be10cf5b51480e57a187d1a16df7a",1,true,true]
 TRACE bwt::indexer                      > fetched 0 transactions between blocks 742456 and 742456
 DEBUG bitcoincore_rpc                   > JSON-RPC request: getblockhash [742456]
 DEBUG bitcoincore_rpc                   > JSON-RPC request: listsinceblock ["00000000000000000005481f63cac68d8c5be10cf5b51480e57a187d1a16df7a",1,true,true]
 TRACE bwt::indexer                      > fetched 0 transactions between blocks 742456 and 742456
 DEBUG bitcoincore_rpc                   > JSON-RPC request: getblockhash [742456]
 DEBUG bitcoincore_rpc                   > JSON-RPC request: listsinceblock ["00000000000000000005481f63cac68d8c5be10cf5b51480e57a187d1a16df7a",1,true,true]
. . .

[craigraw]

You can see at the end it tries to fetch transactions for that block on and on.

@shesek any thoughts on why BWT is making the same requests over and over?