search

lightninglabs / taproot-assets

A layer 1 daemon, for the Taproot Assets Protocol specification, written in Go (golang)
MIT License
439 stars 106 forks source link

Polar Test Flows Working/Not Working #942

Closed levmi closed 1 week ago

levmi commented 2 weeks ago

(h/t to @jamaljsr for running these tests and documenting it all) Jamal ran through test flows on Polar to exhaustively test what is currently working with the latest vs. what is not working. Those flows and their outcomes are outlined below.

The network structure is the following with all litd nodes except for erin which is Core Lightning. I've re-tested using the latest litd branch 0-19-staging (https://github.com/lightninglabs/lightning-terminal/commit/980ee938aa1e7a45633216aff8775ca8d05c12aa)

alice --[asset]-> bob --[sats]-> carol --[asset]-> dave
                                   |
                                 [sats]
                                   |
                                   v
                                  erin (core-ln)

Working

Not Working

Let us know if there are other flows to run through!

Roasbeef commented 2 weeks ago

send asset keysend payment from alice to dave (no_route)

So this can happen if the sender doesn't have a node announcement from the dest yet. In the near future, we'll start to just assume they have the bit set.

Roasbeef commented 2 weeks ago

create asset invoice on alice, pay with asset on dave (multi-hop, opposite direction after funding sending half asset amt. FAILURE_REASON_NO_ROUTE)

In this case, do all parties have enough to satisfy the normal BTC reserve?

jamaljsr commented 2 weeks ago

So this can happen if the sender doesn't have a node announcement from the dest yet. In the near future, we'll start to just assume they have the bit set.

I'll try again, keeping an eye out for the node announcements. Is there anything specific I should be looking for in the logs or describegraph?

In this case, do all parties have enough to satisfy the normal BTC reserve?

Sorry, I forgot to mention that the channel was funded with push_sat=5000. The invoice payment succeeds if paid by a direct peer (bob) but fails for multi-hop. Maybe this is due to the missing node announcement as well. I'll try again.

jamaljsr commented 2 weeks ago

I spent some more time testing these specific failures.

send asset keysend payment from alice to dave (no_route)

So this can happen if the sender doesn't have a node announcement from the dest yet. In the near future, we'll start to just assume they have the bit set.

After noodling on this one a bit more, I'm thinking multi-hop asset->asset keysend payments over BTC channels shouldn't be possible. With an invoice payment, dave would tell carol to swap the incoming sats to the asset in order to receive on their asset channel. For a keysend payments, how would carol know what to do with the incoming payment to dave other than forward sats.

Actually, before it gets to the delivery issue, alice doesn't even know dave exists without the hophint since there's only the private asset channel between carol and dave. So dave doesn't show up in alice's view of the graph. I tested by adding a BTC channel between carol and dave. Now dave is present in alice's graph and when I sent an asset keysend payment from alice to dave, it succeeds. However, it delivers sats to dave over the BTC channel. It makes total sense to me why this happens and doesn't feel like a bug.

Correct me if I'm wrong, but I believe the original use case I was testing (asset->asset multi-hop keysend) is technically not feasible, right?

create asset invoice on alice, pay with asset on dave (multi-hop, opposite direction after funding sending half asset amt. FAILURE_REASON_NO_ROUTE)

In this case, do all parties have enough to satisfy the normal BTC reserve?

The problem with this one turned out to be that the sats channel between bob and carol didn't have enough on carol's side to make the payment in the other direction. After balancing that channel, invoice payments between alice and dave succeed consistently in both directions.

@levmi I think it's safe to remove these two or mark them as resolved in the OP.

levmi commented 2 weeks ago

Marked those 2 as covered!

Roasbeef commented 2 weeks ago

For the direct peer sats -> sats, I think it might be the external traffic shaper getting in the way, was it a keysend payment?

Related-ish logs: 

 2024-06-12 16:12:58.956 [DBG] CRTR: Payment cab2b64d68edef70f4632daed02d249fa73c5e1b33991a1ac55c3c7cb7300513: status=Initiated, active_shards=0, rem_value=10000000 mSAT, fee_limit=0 mSAT                                                                                                                                                                                                                                                                                    
2024-06-12 16:12:58.956 [DBG] CRTR: PaymentSession(cab2b64d68edef70f4632daed02d249fa73c5e1b33991a1ac55c3c7cb7300513): pathfinding for amt=10000000 mSAT                                                                                                                                                                                                                                                                                                                       
2024-06-12 16:12:58.957 [DBG] CRTR: ShortChannelID=463:1:0: external traffic shaper is handling traffic: true                                                                                                                                                                                                                                                                                                                                                                  
2024-06-12 16:12:58.957 [DBG] CRTR: ShortChannelID=463:1:0: external traffic shaper reported available bandwidth: 0 mSAT                                                                                                                                                                                                                                                                                                                                                       
2024-06-12 16:12:58.957 [WRN] CRTR: Not enough outbound balance to send htlc of amount: 10000000 mSAT, only have local balance: 0 mSAT
jamaljsr commented 2 weeks ago

@Roasbeef the sats -> sats payments I tested were using invoices.

jamaljsr commented 2 weeks ago

I've come across an issue where litd will hang when paying invoices for a large asset amounts. I am using the flag --taproot-assets.experimental.rfq.mockoraclecentpersat=1000000 so that the mock exchange rate is 100 sats per 1 asset. The network graph is the same as above, but I minted 1,000,000 AAA assets and funded the asset channels with 400,000 AAA. I funded the sats channel with 10M sats.

I created an invoice on dave for 5,000 AAA (equivalent to 500,000 sats). When I attempt to pay this invoice on alice, the payinvoice command will hang with the output below:

litd@alice:/$ litcli --macaroonpath ~/.lnd/data/chain/bitcoin/regtest/admin.macaroon ln payinvoice -f -asset_id bc8ccb6108f132e00776d2d260997500d9223c7df7923bae71ff71257abd8f93 lnbcrt5m1pnxk9y4pp59shpvhwq7nh7nl5z30v2zy634mdmy9qauj9w6mwshhq8cq4saz4sdqqcqzzsxqyz5vqrzjq2mecn7fxjfjjgn0s284l0zpexrmq8apkakhmu93wfscpcq4dadx4xz3hkc0f2em35qqqqlgqqqqqqgq2qsp5602rr4c5r0xhqprueh5s7shn63dwlcf4c0vd6hme9mc40xt33req9qxpqysgqjrq0hmv8pvnrf5pyctnpu5fkt7jph88cndnzm6je59a7llmcaqdy09m7gfg3skhffvpceu68ff7vc6mckvc32q8wc06h8gtktcahp8spy2sjpf
Asking peer 0344e6f5eef4af071f1d4f867bf0b745a41c2f00d43b238592d426a4777146ea0c for quote to sell assets to pay for invoice over 500000000 msats; waiting up to 60s
Got quote for 5000 asset units at 100000 msat/unit from peer 0344e6f5eef4af071f1d4f867bf0b745a41c2f00d43b238592d426a4777146ea0c with SCID 11993423829610654351
+------------+--------------+--------------+--------------+-------+----------+-----------------+--------------------+
| HTLC_STATE | ATTEMPT_TIME | RESOLVE_TIME | RECEIVER_AMT | FEE   | TIMELOCK | CHAN_OUT        | ROUTE              |
+------------+--------------+--------------+--------------+-------+----------+-----------------+--------------------+
| IN_FLIGHT  |        0.019 |            - | 62500        | 2.124 |      412 | 133040907026432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.042 |            - | 54687.5      | 2.108 |      412 | 133040907026432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.063 |            - | 47851.562    | 2.094 |      412 | 133040907026432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.118 |            - | 41870.117    | 2.082 |      412 | 133040907026432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.136 |            - | 73272.705    | 2.146 |      412 | 133040907026432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.153 |            - | 54954.529    | 2.108 |      412 | 133040907026432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.241 |            - | 41215.896    | 2.082 |      412 | 133040907026432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.260 |            - | 61823.845    | 2.122 |      412 | 133040907026432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.381 |            - | 61823.846    | 2.122 |      412 | 133040907026432 | bob->carol->03eb01 |
+------------+--------------+--------------+--------------+-------+----------+-----------------+--------------------+
Amount + fee:   0 + 0 sat
Payment hash:   2c2e165dc0f4efe9fe828bd8a11351aedbb2141de48aed6dd0bdc07c02b0e8ab
Payment status: IN_FLIGHT

The command does not timeout after the default 60 seconds. I also noticed that after running this command, litd will not shutdown gracefully. Docker has to kill the process in order to stop the container.

I also created a new network then tried a direct peer payment by creating the 5,000 AAA invoice on bob. The behavior is the same in this case as well.

I have tested asset invoice payments of 1K, 2K, 3K, and 4K. Those all succeed. It hangs when I send a payment of 5K or more.

Here are the logs for all of the nodes if it is helpful. alice.log bob.log carol.log dave.log

jamaljsr commented 1 week ago

I believe I have been able to reproduce the above issue in the litd itests by making the following changes to litd_custom_channels_test.go:

Here's the diff of the changes I made to the itests

diff --git a/itest/litd_custom_channels_test.go b/itest/litd_custom_channels_test.go
index 63cc0ef7..01037ab9 100644
--- a/itest/litd_custom_channels_test.go
+++ b/itest/litd_custom_channels_test.go
@@ -53,7 +53,7 @@ var (
        AssetType: taprpc.AssetType_NORMAL,
        Name:      "itest-asset-cents",
        AssetMeta: dummyMetaData,
-       Amount:    1_000_000,
+       Amount:    2_000_000,
    }

    shortTimeout = time.Second * 5
@@ -197,7 +197,7 @@ func testCustomChannels(_ context.Context, net *NetworkHarness,
    // We need to send some assets to Dave, so he can fund an asset channel
    // with Yara.
    const (
-       fundingAmount = 50_000
+       fundingAmount = 400_000
        startAmount   = fundingAmount * 2
    )
    daveAddr, err := daveTap.NewAddr(ctxb, &taprpc.NewAddrRequest{
@@ -410,7 +410,7 @@ func testCustomChannels(_ context.Context, net *NetworkHarness,
    // a direct channel invoice payment with an RFQ SCID present in the
    // invoice.
    // ------------
-   const daveInvoiceAssetAmount = 2_000
+   const daveInvoiceAssetAmount = 60_000
    invoiceResp := createAssetInvoice(
        t.t, charlie, dave, daveInvoiceAssetAmount, assetID,
    )
jamaljsr commented 1 week ago

There's an issue which I previously reported in chat that I forgot to mention here.

After opening a channel with a capacity that's lower than the node's total asset balance, the remaining balance becomes inaccessible. tapcli assets list returns no assets and tapcli assets balance shows the asset with the pre-channel balance.

Expand for reproduction of the issue via CLI ```sh litd@alice:/$ litcli --version litcli version 0.13.0-alpha commit=v0.13.0-alpha-9-g980ee938aa1e7a45633216aff8775ca8d05c12aa litd@alice:/$ tapcli --version tapcli version 0.3.2-alpha commit=v0.3.3-0.20240609154533-2b3c60ae77a9 litd@alice:/$ tapcli assets mint --name PBUX --type normal --supply 1000 { "pending_batch": { "batch_key": "03f44cb28755bb1069a3b44d4af3cc49bbe9e433d56bd1aa05d49efbc997e91aa3", "batch_txid": "", "state": "BATCH_STATE_PENDING", "assets": [ { "asset_version": "ASSET_VERSION_V0", "asset_type": "NORMAL", "name": "PBUX", "asset_meta": null, "amount": "1000", "new_grouped_asset": false, "group_key": "", "group_anchor": "", "group_internal_key": null, "group_tapscript_root": "", "script_key": { "pub_key": "3573758b2c7f60b6c6be2cf7f99d886c1d0d56a53da11e30a96eb926a63a3886", "key_desc": { "raw_key_bytes": "031a620843e9a4a8fdfa76bd27de4d118db6e2d45b23d408802d39057109e783e4", "key_loc": { "key_family": 212, "key_index": 0 } }, "tap_tweak": "" } } ], "created_at": "1718732824", "height_hint": 107, "batch_psbt": "" } } litd@alice:/$ tapcli assets mint finalize { "batch": { "batch_key": "03f44cb28755bb1069a3b44d4af3cc49bbe9e433d56bd1aa05d49efbc997e91aa3", "batch_txid": "19af46087e3c014a0435dc66218563231bdb4c73da9423ea681d15f51c261fda", "state": "BATCH_STATE_BROADCAST", "assets": [ { "asset_version": "ASSET_VERSION_V0", "asset_type": "NORMAL", "name": "PBUX", "asset_meta": null, "amount": "1000", "new_grouped_asset": false, "group_key": "", "group_anchor": "", "group_internal_key": null, "group_tapscript_root": "", "script_key": { "pub_key": "3573758b2c7f60b6c6be2cf7f99d886c1d0d56a53da11e30a96eb926a63a3886", "key_desc": { "raw_key_bytes": "031a620843e9a4a8fdfa76bd27de4d118db6e2d45b23d408802d39057109e783e4", "key_loc": { "key_family": 212, "key_index": 0 } }, "tap_tweak": "" } } ], "created_at": "1718732824", "height_hint": 107, "batch_psbt": "70736274ff0100890200000001cee728c17955bddb96fb686abb2dea74b50dba9690aac4cc5ed149cf6370ce0d00000000000000000002e803000000000000225120bf607a5ae2ecf3e0885fd80d0bb8d3c55c81726b9816ef14a90de3e48fee92f52b1e0f00000000002251202f3851fdcc740d758a5fd17fe3c24b0a636113966d5a3cc7792a8c06c0a965bf00000000000100de02000000000101465ab280735889735105a511dc31eaacfd60b49ca6346cb80c2b97c691a5509f0000000000fdffffff0240420f0000000000160014b71899fb99beb61413d8a1833916464b2160762fbca4f629010000001600141fad0c5c9b6959723abdc3e1d4d6c0560ce1fa0402473044022062d430d4fdfca0f3716e1b47f96d18ca38c19e17692f4ae96e4ac787c1f5a088022076c7c356ef4ea7c426db509b50e1172e80fb58c74f8e41dd6c27db202c165edd012102858ae48c2698bc5c3435e11e799637031d690e82b50c8e29176dc65374d520b36500000001011f40420f0000000000160014b71899fb99beb61413d8a1833916464b2160762f01086c0248304502210097f4e4726a42581f0c05185688eb331b10c530e6be10965f7771a5a2dd26e85e022072eb72eca3dbfe28da288575d127094d240342159ccc31c2e083010cce9feaf40121038f99fbf00e2169d3c29304b0157bf5aa0426a865ec4f2f4ece4320667fba8cb2000022020265d51e54cd1c1aefcf4eaddf74c06d58d5cdb2bfc21b534f6a153319cdf350911800000000560000800000008000000080010000000000000001052065d51e54cd1c1aefcf4eaddf74c06d58d5cdb2bfc21b534f6a153319cdf35091210765d51e54cd1c1aefcf4eaddf74c06d58d5cdb2bfc21b534f6a153319cdf35091190000000000560000800000008000000080010000000000000000" } } # 6 blocks mined here litd@alice:/$ tapcli assets list { "assets": [ { "version": "ASSET_VERSION_V0", "asset_genesis": { "genesis_point": "0dce7063cf49d15eccc4aa9096ba0db574ea2dbb6a68fb96dbbd5579c128e7ce:0", "name": "PBUX", "meta_hash": "0000000000000000000000000000000000000000000000000000000000000000", "asset_id": "4c220f9f2db413a3134dbc98ca0ffb6fa596e6a50a5fa86051625044ed00a899", "asset_type": "NORMAL", "output_index": 0 }, "amount": "1000", "lock_time": 0, "relative_lock_time": 0, "script_version": 0, "script_key": "023573758b2c7f60b6c6be2cf7f99d886c1d0d56a53da11e30a96eb926a63a3886", "script_key_is_local": true, "asset_group": null, "chain_anchor": { "anchor_tx": "02000000000101cee728c17955bddb96fb686abb2dea74b50dba9690aac4cc5ed149cf6370ce0d00000000000000000002e803000000000000225120bf607a5ae2ecf3e0885fd80d0bb8d3c55c81726b9816ef14a90de3e48fee92f52b1e0f00000000002251202f3851fdcc740d758a5fd17fe3c24b0a636113966d5a3cc7792a8c06c0a965bf0248304502210097f4e4726a42581f0c05185688eb331b10c530e6be10965f7771a5a2dd26e85e022072eb72eca3dbfe28da288575d127094d240342159ccc31c2e083010cce9feaf40121038f99fbf00e2169d3c29304b0157bf5aa0426a865ec4f2f4ece4320667fba8cb200000000", "anchor_block_hash": "22eb84a9ef36d099dfe4de67fbf2f73bd2fc91e408105c6f2c4c130b43a44d7f", "anchor_outpoint": "19af46087e3c014a0435dc66218563231bdb4c73da9423ea681d15f51c261fda:0", "internal_key": "03f44cb28755bb1069a3b44d4af3cc49bbe9e433d56bd1aa05d49efbc997e91aa3", "merkle_root": "c65a298b0ec5e8a182370b2641572055c6fccede2732cb36d3ecf2c22ca0b394", "tapscript_sibling": "", "block_height": 0 }, "prev_witnesses": [], "is_spent": false, "lease_owner": "", "lease_expiry": "0", "is_burn": false, "script_key_declared_known": false, "script_key_has_script_path": false } ] } litd@alice:/$ tapcli assets balance { "asset_balances": { "4c220f9f2db413a3134dbc98ca0ffb6fa596e6a50a5fa86051625044ed00a899": { "asset_genesis": { "genesis_point": "0dce7063cf49d15eccc4aa9096ba0db574ea2dbb6a68fb96dbbd5579c128e7ce:0", "name": "PBUX", "meta_hash": "0000000000000000000000000000000000000000000000000000000000000000", "asset_id": "4c220f9f2db413a3134dbc98ca0ffb6fa596e6a50a5fa86051625044ed00a899", "asset_type": "NORMAL", "output_index": 0 }, "balance": "1000" } }, "asset_group_balances": {} } litd@alice:/$ litcli --macaroonpath ~/.lnd/data/chain/bitcoin/regtest/admin.macaroon ln fundchannel --asset_id 4c220f9f2db413a3134dbc98ca0ffb6fa596e6a50a5fa86051625044ed00a899 --node_key 02e61ab7bd13a3df8c8bb5aca2fa976b5bbcdc88afb1c6a8c02184cd86f0fd9615 --sat_per_vbyte 10 --asset_amount 400 { "txid": "ac9ec6252d772dfab1fd9969113702257561b4ab4fddf5f47df3488c2854b032" } # 6 blocks mined here litd@alice:/$ tapcli assets list { "assets": [] } litd@alice:/$ tapcli assets balance { "asset_balances": { "4c220f9f2db413a3134dbc98ca0ffb6fa596e6a50a5fa86051625044ed00a899": { "asset_genesis": { "genesis_point": "0dce7063cf49d15eccc4aa9096ba0db574ea2dbb6a68fb96dbbd5579c128e7ce:0", "name": "PBUX", "meta_hash": "0000000000000000000000000000000000000000000000000000000000000000", "asset_id": "4c220f9f2db413a3134dbc98ca0ffb6fa596e6a50a5fa86051625044ed00a899", "asset_type": "NORMAL", "output_index": 0 }, "balance": "1000" } }, "asset_group_balances": {} } litd@alice:/$ lncli listchannels { "channels": [ { "active": true, "remote_pubkey": "02e61ab7bd13a3df8c8bb5aca2fa976b5bbcdc88afb1c6a8c02184cd86f0fd9615", "channel_point": "ac9ec6252d772dfab1fd9969113702257561b4ab4fddf5f47df3488c2854b032:0", "chan_id": "125344325632000", "capacity": "100000", "local_balance": "96920", "remote_balance": "0", "commit_fee": "2750", "commit_weight": "614", "fee_per_kw": "2500", "unsettled_balance": "0", "total_satoshis_sent": "0", "total_satoshis_received": "0", "num_updates": "0", "pending_htlcs": [], "csv_delay": 144, "private": true, "initiator": true, "chan_status_flags": "ChanStatusDefault", "local_chan_reserve_sat": "1000", "remote_chan_reserve_sat": "1062", "static_remote_key": false, "commitment_type": "SIMPLE_TAPROOT", "lifetime": "106", "uptime": "106", "close_address": "", "push_amount_sat": "0", "thaw_height": 0, "local_constraints": { "csv_delay": 144, "chan_reserve_sat": "1000", "dust_limit_sat": "354", "max_pending_amt_msat": "99000000", "min_htlc_msat": "1", "max_accepted_htlcs": 483 }, "remote_constraints": { "csv_delay": 144, "chan_reserve_sat": "1062", "dust_limit_sat": "354", "max_pending_amt_msat": "99000000", "min_htlc_msat": "1", "max_accepted_htlcs": 483 }, "alias_scids": [ "17592186044416000000" ], "zero_conf": false, "zero_conf_confirmed_scid": "0", "peer_alias": "bob", "peer_scid_alias": "17592186044416000000", "memo": "", "custom_channel_data": { "assets": [ { "asset_utxo": { "version": 1, "asset_genesis": { "genesis_point": "0dce7063cf49d15eccc4aa9096ba0db574ea2dbb6a68fb96dbbd5579c128e7ce:0", "name": "PBUX", "meta_hash": "0000000000000000000000000000000000000000000000000000000000000000", "asset_id": "4c220f9f2db413a3134dbc98ca0ffb6fa596e6a50a5fa86051625044ed00a899" }, "amount": 400, "script_key": "0250aaeb166f4234650d84a2d8a130987aeaf6950206e0905401ee74ff3f8d18e6" }, "capacity": 400, "local_balance": 400, "remote_balance": 0 } ] } } ] } litd@alice:/$ ```
ffranr commented 1 week ago

I've come across an issue where litd will hang when paying invoices for a large asset amounts. I am using the flag --taproot-assets.experimental.rfq.mockoraclecentpersat=1000000 so that the mock exchange rate is 100 sats per 1 asset. The network graph is the same as above, but I minted 1,000,000 AAA assets and funded the asset channels with 400,000 AAA. I funded the sats channel with 10M sats.

I created an invoice on dave for 5,000 AAA (equivalent to 500,000 sats). When I attempt to pay this invoice on alice, the payinvoice command will hang with the output below:

litd@alice:/$ litcli --macaroonpath ~/.lnd/data/chain/bitcoin/regtest/admin.macaroon ln payinvoice -f -asset_id bc8ccb6108f132e00776d2d260997500d9223c7df7923bae71ff71257abd8f93 lnbcrt5m1pnxk9y4pp59shpvhwq7nh7nl5z30v2zy634mdmy9qauj9w6mwshhq8cq4saz4sdqqcqzzsxqyz5vqrzjq2mecn7fxjfjjgn0s284l0zpexrmq8apkakhmu93wfscpcq4dadx4xz3hkc0f2em35qqqqlgqqqqqqgq2qsp5602rr4c5r0xhqprueh5s7shn63dwlcf4c0vd6hme9mc40xt33req9qxpqysgqjrq0hmv8pvnrf5pyctnpu5fkt7jph88cndnzm6je59a7llmcaqdy09m7gfg3skhffvpceu68ff7vc6mckvc32q8wc06h8gtktcahp8spy2sjpf
Asking peer 0344e6f5eef4af071f1d4f867bf0b745a41c2f00d43b238592d426a4777146ea0c for quote to sell assets to pay for invoice over 500000000 msats; waiting up to 60s
Got quote for 5000 asset units at 100000 msat/unit from peer 0344e6f5eef4af071f1d4f867bf0b745a41c2f00d43b238592d426a4777146ea0c with SCID 11993423829610654351
+------------+--------------+--------------+--------------+-------+----------+-----------------+--------------------+
| HTLC_STATE | ATTEMPT_TIME | RESOLVE_TIME | RECEIVER_AMT | FEE   | TIMELOCK | CHAN_OUT        | ROUTE              |
+------------+--------------+--------------+--------------+-------+----------+-----------------+--------------------+
| IN_FLIGHT  |        0.019 |            - | 62500        | 2.124 |      412 | 133040907026432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.042 |            - | 54687.5      | 2.108 |      412 | 133040907026432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.063 |            - | 47851.562    | 2.094 |      412 | 133040907026432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.118 |            - | 41870.117    | 2.082 |      412 | 133040907026432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.136 |            - | 73272.705    | 2.146 |      412 | 133040907026432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.153 |            - | 54954.529    | 2.108 |      412 | 133040907026432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.241 |            - | 41215.896    | 2.082 |      412 | 133040907026432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.260 |            - | 61823.845    | 2.122 |      412 | 133040907026432 | bob->carol->03eb01 |
| IN_FLIGHT  |        0.381 |            - | 61823.846    | 2.122 |      412 | 133040907026432 | bob->carol->03eb01 |
+------------+--------------+--------------+--------------+-------+----------+-----------------+--------------------+
Amount + fee:   0 + 0 sat
Payment hash:   2c2e165dc0f4efe9fe828bd8a11351aedbb2141de48aed6dd0bdc07c02b0e8ab
Payment status: IN_FLIGHT

The command does not timeout after the default 60 seconds. I also noticed that after running this command, litd will not shutdown gracefully. Docker has to kill the process in order to stop the container.

I also created a new network then tried a direct peer payment by creating the 5,000 AAA invoice on bob. The behavior is the same in this case as well.

I have tested asset invoice payments of 1K, 2K, 3K, and 4K. Those all succeed. It hangs when I send a payment of 5K or more.

Here are the logs for all of the nodes if it is helpful. alice.log bob.log carol.log dave.log

I think the above bug is solved by https://github.com/lightninglabs/taproot-assets/pull/959 I'll aim to get it merged and then bump the litd staging branch so that we can test once more.

ffranr commented 1 week ago

Closing this issue as incomplete tasks have been rolled into their own issues.