search

lightningnetwork / lnd

Lightning Network Daemon ⚡️
MIT License
7.7k stars 2.09k forks source link

[bug]: subscribe transactions #7137

Closed saraogiraj94 closed 1 year ago

saraogiraj94 commented 2 years ago

Background

Hi found an issue in https://api.lightning.community/?javascript#subscribetransactions Below you can find messages for tx_hash : 7ce5cdae3212bb69b1ac659794dfee6c9ec1c0db61196177a59d5b02f6cc92c5 Now the timestamp of num_confirmations : 1 is 1668055807 num_confirmations : 0 is 1668055824 How can timestamp of 0 confirmation be greater the timestamp of 1 confirmation, ideally it should be less and difference should be the time taken to do confirmation. Bcs of this in subscribe transactions I am getting messages when 1st confirmation happen, despite setting end_height as -1. { "dest_addresses": [ 'tb1q7u39m03acj4ys7qmx5jzgu3cznpt8shp29m5an', 'tb1qwgpxl9fxp70f6rwpa4r47g5uklemsettjmadfj' ], output_details: [ { output_type: 'SCRIPT_TYPE_WITNESS_V0_PUBKEY_HASH', address: 'tb1q7u39m03acj4ys7qmx5jzgu3cznpt8shp29m5an', pk_script: '0014f7225dbe3dc4aa48781b352424723814c2b3c2e1', output_index: '0', amount: '1000', is_our_address: true }, { output_type: 'SCRIPT_TYPE_WITNESS_V0_PUBKEY_HASH', address: 'tb1qwgpxl9fxp70f6rwpa4r47g5uklemsettjmadfj', pk_script: '001472026f95260f9e9d0dc1ed475f229cb7f3b8656b', output_index: '1', amount: '362', is_our_address: false } ], previous_outpoints: [ { outpoint: '27e00f4fb2406d23dcb8979e67f3d20f1530353b1606a329991a7083dbf9391d:0', is_our_output: false } ], tx_hash: '7ce5cdae3212bb69b1ac659794dfee6c9ec1c0db61196177a59d5b02f6cc92c5', amount: '1000', num_confirmations: 0, block_hash: '', block_height: 0, time_stamp: '1668055824', total_fees: '0', raw_tx_hex: '020000000001011d39f9db83701a9929a306163b3530150fd2f3679e97b8dc236d40b24f0fe0270000000000feffffff02e803000000000000160014f7225dbe3dc4aa48781b352424723814c2b3c2e16a0100000000000016001472026f95260f9e9d0dc1ed475f229cb7f3b8656b024730440220797b2b2e815b63463edecd514ec8ad59609b9ae650719111cb755d82c68a131c02205c2d270eeae5578ede84cd18ed9cddeae02872da388a84656bde23b598de60c80121039c4fa5f8a2028e6c3e405a1f0e44b79423581e474e53fef91694347ab86f445befb42400', label: '' } { dest_addresses: [ 'tb1q7u39m03acj4ys7qmx5jzgu3cznpt8shp29m5an', 'tb1qwgpxl9fxp70f6rwpa4r47g5uklemsettjmadfj' ], output_details: [ { output_type: 'SCRIPT_TYPE_WITNESS_V0_PUBKEY_HASH', address: 'tb1q7u39m03acj4ys7qmx5jzgu3cznpt8shp29m5an', pk_script: '0014f7225dbe3dc4aa48781b352424723814c2b3c2e1', output_index: '0', amount: '1000', is_our_address: true }, { output_type: 'SCRIPT_TYPE_WITNESS_V0_PUBKEY_HASH', address: 'tb1qwgpxl9fxp70f6rwpa4r47g5uklemsettjmadfj', pk_script: '001472026f95260f9e9d0dc1ed475f229cb7f3b8656b', output_index: '1', amount: '362', is_our_address: false } ], previous_outpoints: [ { outpoint: '27e00f4fb2406d23dcb8979e67f3d20f1530353b1606a329991a7083dbf9391d:0', is_our_output: false } ], tx_hash: '7ce5cdae3212bb69b1ac659794dfee6c9ec1c0db61196177a59d5b02f6cc92c5', amount: '1000', num_confirmations: 1, block_hash: '000000000000002b3889851c7ed22404acaa7aba492ea1bec8a1eb4795dc18ca', block_height: 2405616, time_stamp: '1668055807', total_fees: '0', raw_tx_hex: '020000000001011d39f9db83701a9929a306163b3530150fd2f3679e97b8dc236d40b24f0fe0270000000000feffffff02e803000000000000160014f7225dbe3dc4aa48781b352424723814c2b3c2e16a0100000000000016001472026f95260f9e9d0dc1ed475f229cb7f3b8656b024730440220797b2b2e815b63463edecd514ec8ad59609b9ae650719111cb755d82c68a131c02205c2d270eeae5578ede84cd18ed9cddeae02872da388a84656bde23b598de60c80121039c4fa5f8a2028e6c3e405a1f0e44b79423581e474e53fef91694347ab86f445befb42400', label: '' }

Your environment

Steps to reproduce

Tell us how to reproduce this issue. Please provide stacktraces and links to code in question.

Expected behaviour

Tell us what should happen

Actual behaviour

Tell us what happens instead

GeorgeTsagk commented 2 years ago

Hey @saraogiraj94

I looked up your TX and seems to be on testnet, another thing I noticed is that the block after the one your TX was included in got mined within ~1min. Keep in mind that miners are the ones who set the timestamp of the block, so it could be an issue of inaccuracy by the 2nd miner.

saraogiraj94 commented 1 year ago

I tested on mainnet node still the issue persist, got both 0 and 1 confirmation together

    "txHash": "2f24b8fc195950de4815c5789347c7b752ece5aee9fac7a2d566d20d4147e383",
    "amount": 10000,
    "numConfirmations": 0,
    "blockHash": "",
    "blockHeight": 0,
    "timeStamp": 1668419349,
    "totalFees": 0,
    "destAddresses": [
        "bc1qqwq0xpe9m797je9vk6zafvnf9xqdgc9kzewj7u",
        "bc1qxc0afdm4eh56zjsrukv4cafajd8j2ps49sv3tu"
    ],
    "outputDetails": [
        {
            "outputType": "SCRIPT_TYPE_WITNESS_V0_PUBKEY_HASH",
            "address": "bc1qqwq0xpe9m797je9vk6zafvnf9xqdgc9kzewj7u",
            "pkScript": "00140380f30725df8be964acb685d4b2692980d460b6",
            "outputIndex": 0,
            "amount": 999133,
            "isOurAddress": false
        },
        {
            "outputType": "SCRIPT_TYPE_WITNESS_V0_PUBKEY_HASH",
            "address": "bc1qxc0afdm4eh56zjsrukv4cafajd8j2ps49sv3tu",
            "pkScript": "0014361fd4b775cde9a14a03e5995c753d934f250615",
            "outputIndex": 1,
            "amount": 10000,
            "isOurAddress": true
        }
    ],
    "rawTxHex": "02000000000101a2ff7a168cb00befc06b97cb442a4e7f905c6b03c2f6c8340c6f806d20d21fa30100000000feffffff02dd3e0f00000000001600140380f30725df8be964acb685d4b2692980d460b61027000000000000160014361fd4b775cde9a14a03e5995c753d934f250615024730440220589166ac7fc25789aa4addfb7f533e13c60cd381c263ba5bf74106de2c2a97460220348ed2f9a614273eebb846695201456f3d4bede267bea7b962a4dc18e09081040121029fb7cef942ad68d38d2181d002fdaf6120cc1f84bf13ecaf5ddf37c9de66b4a600000000",
    "label": "",
    "previousOutpoints": [
        {
            "outpoint": "a31fd2206d806f0c34c8f6c2036b5c907f4e2a44cb976bc0ef0bb08c167affa2:1",
            "isOurOutput": false
        }
    ]
}
    "txHash": "2f24b8fc195950de4815c5789347c7b752ece5aee9fac7a2d566d20d4147e383",
    "amount": 10000,
    "numConfirmations": 1,
    "blockHash": "0000000000000000000465a35039a13801563c5947a2d3d0defc013b4733a759",
    "blockHeight": 763129,
    "timeStamp": 1668419309,
    "totalFees": 0,
    "destAddresses": [
        "bc1qqwq0xpe9m797je9vk6zafvnf9xqdgc9kzewj7u",
        "bc1qxc0afdm4eh56zjsrukv4cafajd8j2ps49sv3tu"
    ],
    "outputDetails": [
        {
            "outputType": "SCRIPT_TYPE_WITNESS_V0_PUBKEY_HASH",
            "address": "bc1qqwq0xpe9m797je9vk6zafvnf9xqdgc9kzewj7u",
            "pkScript": "00140380f30725df8be964acb685d4b2692980d460b6",
            "outputIndex": 0,
            "amount": 999133,
            "isOurAddress": false
        },
        {
            "outputType": "SCRIPT_TYPE_WITNESS_V0_PUBKEY_HASH",
            "address": "bc1qxc0afdm4eh56zjsrukv4cafajd8j2ps49sv3tu",
            "pkScript": "0014361fd4b775cde9a14a03e5995c753d934f250615",
            "outputIndex": 1,
            "amount": 10000,
            "isOurAddress": true
        }
    ],
    "rawTxHex": "02000000000101a2ff7a168cb00befc06b97cb442a4e7f905c6b03c2f6c8340c6f806d20d21fa30100000000feffffff02dd3e0f00000000001600140380f30725df8be964acb685d4b2692980d460b61027000000000000160014361fd4b775cde9a14a03e5995c753d934f250615024730440220589166ac7fc25789aa4addfb7f533e13c60cd381c263ba5bf74106de2c2a97460220348ed2f9a614273eebb846695201456f3d4bede267bea7b962a4dc18e09081040121029fb7cef942ad68d38d2181d002fdaf6120cc1f84bf13ecaf5ddf37c9de66b4a600000000",
    "label": "",
    "previousOutpoints": [
        {
            "outpoint": "a31fd2206d806f0c34c8f6c2036b5c907f4e2a44cb976bc0ef0bb08c167affa2:1",
            "isOurOutput": false
        }
    ]
}
sputn1ck commented 1 year ago

I'm not sure I understand the problem. Is it that you don't get the 0 confirmation notification when you publish the transaction, but only when it is confirmed?

guggero commented 1 year ago

Are you using a Neutrino backend?

saraogiraj94 commented 1 year ago

@sputn1ck I got the notification of 0 confirmation with 1 confirmation together. So 0 confirmation should have come as the transaction was initiated, way before 1st confirmation.

@guggero No its bitcoind full node with txindex=1

saraogiraj94 commented 1 year ago

Any update on this ?

sputn1ck commented 1 year ago

I'm having trouble reproducing this issue. I'm running SubscribeTransactions with EndHeight: -1 on node A, getting a new address on node B, then SendCoins from node A to node B. As soon as send the coins I get the first notification from SubscribeTransactions. When I mine a block I get the new notification.

Could you elaborate a bit more on your exact setup and the order of how you call things?

saraogiraj94 commented 1 year ago

In your case, since you are running SubscribeTransactions on Node A : try to generate address from node A and send coins to that address from node B. At that I am facing the above issue.

sputn1ck commented 1 year ago

In your case, since you are running SubscribeTransactions on Node A : try to generate address from node A and send coins to that address from node B.

Similiar to the other way around, I first get the 0 conf notification and then the 1 conf notification once I mine the transaction.

In your setup are both nodes connected to the same bitcoind node?

saraogiraj94 commented 1 year ago

We have tried with same bitcoin node as well as has used, 3rd party aap like phoneix to send the bitcoin to our node address.

guggero commented 1 year ago

Sounds like your connection from lnd to bitcoind isn't working correctly maybe? Can you check your ZMQ settings?

sputn1ck commented 1 year ago

3rd party aap like phoneix to send the bitcoin to our node address.

that would lead me to believe that the transaction never reaches the mempool of your bitcoind, so you only process the transaction once the new block comes in, however

We have tried with same bitcoin node

seems like it should send the 0 conf notification immediatly.

saraogiraj94 commented 1 year ago

@guggero The following is my bitcoin conf dbcache=1536 par=1 blocksonly=1 maxuploadtarget=137 maxconnections=16 printtoconsole=1 testnet=1 [test] rpcuser=bitcoinrpc rpcpassword=bitcoinrpc rpcbind=0.0.0.0 rpcallowip=0.0.0.0/0 server=1 rpcport=8332 zmqpubrawblock=tcp://0.0.0.0:12005 zmqpubrawtx=tcp://0.0.0.0:12006 txindex=1

I run bitcoin inside docker and have mapped its port to host.

My LND conf is : `debuglevel=debug rpclisten=0.0.0.0 restlisten=0.0.0.0:80 listen=0.0.0.0:9736 alias=loclal color=#f2f2f2 maxpendingchannels=5

bitcoin.testnet=1 bitcoind.rpchost=10.0.4.230:8332 bitcoin.active=1 bitcoin.node=bitcoind bitcoind.rpcuser=bitcoinrpc bitcoind.rpcpass=bitcoinrpc

bitcoind.zmqpubrawblock=tcp://10.0.4.230:12005 bitcoind.zmqpubrawtx=tcp://10.0.4.230:12006

watchtower.active=1 wtclient.active=true wallet-unlock-password-file=/tmp/lnd.password wallet-unlock-allow-create=true allow-circular-route=true accept-keysend=true maxlogfiles=100 maxlogfilesize=10 minchansize=100000 tlsextraip=10.0.5.39

db.bolt.auto-compact=true accept-amp=true`

As you can see have added the host in zmq of bitcoind node.

saraogiraj94 commented 1 year ago

that would lead me to believe that the transaction never reaches the mempool of your bitcoind, so you only process the transaction once the new block comes in, however

@sputn1ck how can I verify and rectify this ?

guggero commented 1 year ago

blocksonly=1 means your bitcoind will not forward transactions, only blocks. That's why lnd doesn't get them until they are in a block.

saraogiraj94 commented 1 year ago

@guggero and @sputn1ck you guys are saviour !! Thanks for the update. It worked.

Now have 2 questions as mystery: 1) Now getting 0 and 1 confirmations, but not more then that in subscriberawtransactions. Can you assist how can I get more confirmations. 2) If I am setting start_height to old blocks, then I am not able to fetch the old transactions.

If @guggero and @sputn1ck could assist in this.

guggero commented 1 year ago

I think it makes sense to not give further updates after a transaction was confirmed. Otherwise there might be a huge number of messages on the RPC if the wallet is very active. If you want more fine-grained control over confirmations, look into the ChainNotifier RPC that will give you updates for new blocks arriving or lets you subscribe to a specific TXID to wait for a certain number of transactions.

Closing this issue as the problem was the blocksonly=1 setting in bitcoind that prevented unconfirmed transactions to be relayed to lnd.